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Abstract 



o 

^ ' We study the evolutionary robustness of strategies in infinitely repeated prisoners' dilemma 

games in which players make mistakes with a small probability and are patient. The evolutionary 
process we consider is given by the replicator dynamics. We show that there are strategies with a 
uniformly large basin of attraction independent of the size of the population. Moreover, we show 
that those strategies forgive defections and, assuming that they are symmetric, they cooperate. 
• We provide partial efficiency results for asymmetric strategies. 

q 

^ ■ 1 Introduction 

S . 

The theory of infinitely repeated games has been very influential in the social sciences showing 
how repeated interaction can provide agents with incentives to overcome opportunistic behavior. 
CsJ | However, a usual criticism of this theory is that there may be a multiplicity of equilibria. While 

cooperation can be supported in equilibrium when agents are sufficiently patient, there are also 
equilibria with no cooperation. Moreover, a variety of different punishments can be used to support 
cooperation. 

To solve this multiplicity problem, we study what types of strategies will have a large basin of 



attraction regardless of what other strategies are considered in the evolutionary dynamic. More 
precisely, we study the replicator dynamic over arbitrary finite set of infinitely repeated strategies 



in which the strategy makes a mistake with a small probability 1 — p in every round of the game. 
We study which strategies have a non-vanishing basin of attraction with a uniform size regardless 
of the set of strategies being considered in the population. We say that a strategy has a uniformly 
large basin of attraction if it repels invasions of a given size for arbitrarily patient players and small 
probability of errors and for any possible combination of alternative strategies (see definition [3] for 
details) . 

We find that two well known strategies, "always defect" and "grim," do not have uniformly large 
basins of attraction. Moreover, any strategy that does not forgive cannot have a uniformly large 
basin either. The reason is that, as players become arbitrarily patient and the probability of errors 
becomes small, unforgiving strategies lose in payoffs relative to strategies that forgive and the size 
of the basins of attraction between these two strategies will favor the forgiving one. This is the case 
even when the inefficiencies happen off the equilibrium path (as it is the case for grim). 

Also, we show that symmetric strategies leading to inefficient payoffs (on or off the path) cannot 
have uniformly large basins of attractions. We also provide some efficiency results for asymmetric 
strategies. First, we show that there is a relationship between the size of the basin of attraction 
and the frequency of cooperation. Second, we show that there is a relationship between the degree 
of asymmetry of a strategy and its efficiency. Third, we show that strategies with a uniformly large 
basin of attraction cannot have inefficient payoffs in all histories. 

It could be the case that inefficient and unforgiving strategies do not have uniformly large basins 
since actually there may be no strategies with that property! We prove that that is not the case by 
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showing that the strategy " win-stay- lose-shift" has a uniformly large basin of attraction, provided a 
sufficiently small probability of mistakes. As this strategy is efficient (and symmetric), we show that 
the concept of uniformly large basins of attraction provides a (partial) solution to the long studied 
problem of equilibrium selection in infinitely repeated games: only efficient equilibria survive for 
patient players if we focus on symmetric strategies. 

Note that we not only provide equilibrium selection at the level of payoffs but also at the level of 
the type of strategies used to support those payoffs: the payoffs from mutual cooperation can only 
be supported by strategies that do not involve asymptotically inefficient punishments. This provides 
theoretical support to the claims of Axelrod [AxJ, that successful strategies should be cooperative 
and forgiving. 

In addition, we prove that our results are robust to perturbation of the replicator dynamic 
provided that it is still the case that the only growing strategies are those that perform better than 
the average. 

In our study of the replicator dynamics (and its perturbations) we develop tools that can be 
used to analyze the basins of attractions outside of the particular case of infinitely repeated games. 
In fact the results are based in a series of theorems about general replicator dynamics which can be 
used to study the robustness of steady states for games in general. 

An extensive previous literature has addressed the multiplicity problem in infinitely repeated 
games. Part of this literature focuses on strategies of finite complexity with costs of complexity to 
select a subset of equilibria (see Rubinstein [RJ , Abreu and Rubinstein [ARJ , Binmore and Samuelson 
[BiSJ, Cooper [C] and Volij [Y])- This literature finds that the selection varies with the equilibrium 
concept being used and the type of cost of complexity. Another literature appealed to ideas of 
evolutionary stability as a way to select equilibria and found that no strategy is evolutionary stable 
in the infinitely repeated prisoners' dilemma (Boyd and Lorberbaum |BLJ). The reason is that for 
any strategy there exists another strategy that differs only after events that are not reached by this 
pair of strategies. As such, the payoff from both strategies is equal when playing with each other 
and the original strategy cannot be an attractor of an evolutionary dynamic. Bendor and Swistak 
[BeSJ circumvent the problem of ties by weakening the stability concept and show that cooperative 
and retaliatory strategies are the most robust to invasions. 

In a different approach to ties, Boyd [B] introduced the idea of errors in decision making. If 
there is a small probability of errors in every round, then all events in a game occur with positive 
probability destroying the certainty of ties allowing for some strategies to be evolutionary stable. 
However, as shown by Boyd [BJ and Kim [KiJ, many strategies that are sub-game perfect for a given 
level of patience and errors can also be evolutionary stable. 

Fudenberg and Maskin [FM2J (see also Fudenberg and Maskin [FMJ) show that evolutionary 
stability can have equilibrium selection implications if we ask that the size of invasions that the 
strategy can repel to be uniformly large with respect to any alternative strategy and for large 
discount factors and small probabilities of mistakes. They show that the only strategies with 
this characteristic must be cooperative. There are three main differences with our results. First, 
Fudenberg and Maskin |FM2] focus on strategies of finite complexity while we do not have that 
restriction. Second, our robustness concept does not only consider the robustness to invasion by a 
single alternative strategy but also robustness to invasion by any arbitrary combination of alternative 
strategies. In other words, we also look at the size of the basin of attraction inside the simplex. 
Third, our full efficiency result only applies to the case of symmetric strategies and we only provide 
partial efficiency results for the general case. We want to point out that to prove efficiency we use 
a similar approach to the one used in |FM2] , 

Our results also relate to Johnson, Levine and Pesendorfer |JLP] . Volij [V] and Levine and 
Pesendorfer [LP] who use stochastic stability (Kandori, Mailath and Rob [KMRJ and Young |YPJ) 
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to select equilibria in infinitely repeated games. As having large basin of attraction is a necessary 
condition (but not sufficient) for stochastic stability, the present results could help characterize 
strategies that are stochastically stable in any finite population. 

There is a previous theoretical literature providing evolutionary support for the strategy win- 
stay- lose-shift (see Nowak and Sigmund |NS| and Imhof, Fudenberg and Nowak [IFNJ). This strategy 
has received little support from experiments on infinitely repeated games (see Dal Bo and Frechette 
[DBF], Fudenberg, Rand and Dreber |FRD] and Dal Bo and Frechette [DBF2]). We hope that new 
experiments can be designed to test this strategy's robustness to invasions when it is already highly 
prevalent in the population. 

Finally, our result linking the size of the basin of attraction and the frequency of cooperation 
relates to other experimental evidence provided by Dal B6 and Frechette [DBF ], They find that 
the frequency of cooperation is increasing in the size of the basin of attraction of Grim versus the 
strategy Always Defect. 

The paper is organized as follows: In section [2] we introduce the infinite repeated prisoners' 
dilemma with trembles. In section[3]we define the replicator dynamics in any dimension. In theorem 
[TJwe give sufficient conditions for a vertex to have a large local basin of attraction independent of 
the dimension of the payoff matrix. More precisely, the conditions give a lower bound for the size 
of the basin of attraction. Those conditions are based on comparing simultaneously the dynamics 
associated to group of three vertices. Moreover, in subsection 13 . 41 we show that for a two dimensional 
simplex, the conditions of theorem [JJ with extra assumptions, also provide an upper bound for the 
size of the basin of attraction. In particular, we show that to have a large basin of attraction is not 
enough to analyze the dynamic of one dimensional simplex. In section [5] we recast the replicator 
dynamics in the context of infinite repeated prisoners' dilemma with trembles. We define the notion 
of strategy having a uniformly large basin of attraction (see definition [3]) . In section [5] we show that 
unforgiving strategies do not have a uniformly large basin (this includes the strategies always defect 
and grim). In section [6] we prove that for any history, the frequency of cooperation converges to one 
for symmetric strategies that have a uniformly large basin of attraction. In subsection 16.21 we obtain 
an efficiency result for asymmetric strategies. First, we show that there is a relationship between 
the frequency of cooperation of a strategy and the size of its basin of attraction. Second, we show 
the frequency of cooperation after any history is bounded below by a quantity that is related to 
how asymmetric the strategy involved is (the less asymmetric a strategy is, the higher the frequency 
of cooperation). Third, we show that if there is a finite path such that the strategy does no fully 
cooperate with itself for any sequel history, then the strategy does not have a uniformly large basin 
of attraction (see theorem [7] and observe that this result requires considering populations of at least 
three strategies) . In section [7] we show how to adapt theorem [TJ to the context of the set of all 
the strategies. In particular, in subsection 17. 1 1 we provide sufficient conditions to guarantee that a 
strategy has a uniform large basin of attraction. As they follow from theorem [TJ these conditions 
basically consist of analyzing all the possible set of three strategies. Moreover, in subsection 17.21 we 
show that weaker conditions that consist of comparing sets of two strategies are not enough to have 
a uniformly large basin of attraction. In section [8] we develop a technique to calculate the payoff 
with trembles for certain type of strategies (see definition [XT]) provided certain restriction on the 
probability of mistakes (see lemma [To]) . In section [9] we apply this techniques for the particular case 
of win-stay- lose-shift, proving that it has a uniformly large basin of attraction. We also consider in 
subsection 19.11 a generalization of win-stay- lose-shift. In subsection [10] we show that theorem [JJ also 
holds for a general type of equation that resembles the replicator dynamics. 
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2 Infinitely repeated prisoners' dilemma with trembles 



In the present section, we state the definitions of the game first without trembles and later with 
trembles as in |FM2] , We explain how the payoff are calculated whit and without trembles. 

In each period t = 0, 1, 2, ... the 2 agents play a symmetric stage game with action space A = 
{C, D}. At each period t player one chooses action a* £ A and player two chooses action b l £ A. We 
denote the vector of actions until time t as at = (a , a 1 , . . . , a*) for player one and bt = (b°, b , . . . , b l ) 
for player two. The payoff from the stage game at time t is given by utility function u{a , &*) : 
A x A — > K for player one and it(6*,o ) : A x A — > 5ft for player two such that u(D,C) = T. 
u(C, C) = R, u{D, D) = P, u{C, D) = S, with T > R > P > S and 2R > T + S. 

Agents observe previous actions and this knowledge is summarized by histories. When the game 
begins we have the null history h° = (a , 6°), afterwards ht = (flSt-i, £>t-i) = ({oP, b°), . . . (a* -1 , b 1 ^ 1 )) 
and H l is the space of all possible t histories. Let be the set of all possible histories. A pure 
strategy is a function s : Ut^aHt —> A. In other words, a pure strategy s is a functions s : Ht —> A 
for all t. 

It is important to remark, that given two strategies S\,S2 and a finite path ht = (at-i,bt-i), if 
s\ encounter S2 then 

h* = (s 1 (h t ),s 2 (h t )), 

where 

h t := (6 t _i,a t -i). (1) 

Given a pair of strategies (si, S2) we denote the history that they as h SljS2 . In other words, denoting 
with h sltS2t the path up to period t — 1 then the path h aijS2 , is the path that verifies 

s l{ h si,s 2t ) = a\ s 2 (h SuS2t ) = b\ 

Given a pair of strategies s±, s 2 the utility of the agent playing s± is 

00 

U{s 1 ,s 2 ) = (1-5) ^2S t u(si(h sl>S2t ),s 2 (,hs 1 ,s2t))> 
t=o 

where the common and constant discount factor 8 < 1. 

Given a finite path ht, with ^ sl)S2 //i t we denote the equilibrium path between si and s 2 with 

seed ht Given the recursivity of the discounted utility function we can write the utility starting from 

00 — 

history h t as U(s u s 2 \h t ) = (1 - 5) Yl St ~ ku ( s i( h Sl ,s 2 /h tk ), s 2 (h SljS2 / htk )). 

k=t 

For the case of trembles, we have the probability of making a mistake, more precisely, with a 
positive p < 1 we denote the probability that a strategy perform what intends. Now, given two 
strategies s\, s 2 (they can be the same strategy) we define 

Us, P (s 1 ,s 2 ) = (1 - 5) 5 t p SuS2 {a t ,bt)u(a t ,b t ) 

t^0,a t ,h 

where u(a t ,b t ) denotes the usual payoff of the pair (a*, b f ) and p SliS2 {at,bt) denote the probability 
that the strategies s\ and s 2 go through the path ht = (at,bt) when they are playing one to each 
other. To define p SltS2 (at,bt) we proceed inductively: 

p 81 ,sMM) =p sl , S2 (a t -iM-i)p it+jt (i -p) 1 -*^ 1 -* (2) 

where 
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(i) it = 1 if a* = si(h t ), it = otherwise, 

(ii) jt = 1 if b l = s 2 (h t - 1 ), jt = otherwise. 
Therefore, 

Psi , S2 (a t ,b t ) = P mt+nt {l -p)2t+2-«H-n« 

where 

mt = Cardinal{0 ^ i ^ t : si(hi) = a 1 } 
n t = Cardinal{0 ^ i ^ t : s 2 (hj) = 
Observe that if ht G ^si,s 2 (meaning that /i t = h Sl ^ S2t ) then 

Psi , S2 (/ it )=p 2t+2 . (3) 

With 

C / *,p ) h. 1 ,. 2 (*i.S2) 

we denote the utility only along the path h sl)S2 . With Us tPt h% S2 (si,s 2 ) we denote the difference, 
i.e., Us ! p(s 1 ,s 2 ) - U Sj p jhsiiS2 (s 1 ,s 2 ). Now, given a finite string /i t with 

Us, p (si,s 2 /ht) 
we denote the utility with seed ht and with 

U S,p(h S i,S2/h t ) 

we denote the utility only along the path with seed ht for the pair s±,s 2 . In the same way, with 
Us, P (h c Si S2 / ht ) we denote Us )P (si, s 2 /h t ) — Us :P (h Sl ^ S2 / ht ). Also, with MS we denote the set of path 
which are not h SltS2 — paths; usually those paths are called second order paths. 

Definition 1. We say that s is a subgame perfect strategy if for any s' different than s it follows 
that 

Us, p (s,s/h t ) - U SjP (s',s/h t ) > 0. 
It is also said that s is a strict subgame perfect strategy if Us :P (s, s/h t ) — Us :P (s' , s/ht) > 0. 

Let us consider two strategies si and s 2 and let 

K Sl ,s 2 ■= {h G H : 3 k ^ 0, si(ht) = s 2 (h t ) Vt < k; si(h k ) / s 2 (h k )}. 

Observe that if s±(ho) / s 2 (ho) then any path h G #oo belongs to 1Z Sl ^ 2 . In other words, we 
consider all the paths where s\ and s 2 differ at some moment, including the first move. Observe 
that k depends on h, and it is defined as the first time that si differs with s 2 along h, i.e. 

kh(si, s 2 ) = min{i ^ : si(h t ) / s 2 (h t )}. 

From now on, to avoid notation we drop the dependence on the path, and with h k we denote the 
k— finite truncation of h where k is the first time that s\ and s 2 deviate along h. Observe that for 
h G lZ Sl!S2 , the fact that si(ht) = s 2 (h t ) for any t < k does not imply that h* = si(ht). Moreover, 
observe also that if s\ ^ s 2 then 

K, u , a ± 0. 

From now on, given h G 7£ sljS2 with h k we denote the finite path contained in h such that s\(ht) = 
s 2 {ht) for any t < k and si(h k ) / s 2 (h k ) 
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Lemma 1. It follows that 

Us,p(s 1 ,8i)-Us,p(s 2 ,s 1 ) = ^2 5 k Vs 1 ,s 1 { h k){U8, P {si,si/h k ) - Us tP (s 2 ,si/h k )). (4) 

h k ,heTZ Sl ,s 2 

Proof. If si(ho) 7^ s 2 (ho) then TZ sltS2 = Hq, h k = Hq and in this case there is nothing to prove. If 
si(0) = s 2 (0), the result follows from the next claim that states that given a history path h then 

(h) = I P^.si^t) if 1 < k 

,n 1 Ps 2 ,sA h k)Ps 2 , Sl /hM k (. h )t-k) =Ps 1 ,s 1 (hk)Ps 2 ,s 1 /h k (^ k {h) t -k) ift>k 

(recall that a k (h) is a history path that verifies a k (h)j = hj +k ). To prove the claim in the case that 
t ^ k we proceed by induction: recalling Qfollows that 

p SltSl (a t ,b t ) = ps^iat-uh^)/^ (I - p) 2 - 1 '^ (5) 

where 

(i) i\ = 1 if a t = sx(h t -\) = Si(of_i, 6t_i), i\ = otherwise, 

(ii) jj 1 = 1 if b t = si(ht-i) = si(6 t _i, a t _i), j t x = otherwise 
and 

p S2!S1 (a t , b t ) = p S2tSl {a t -i,b t -i)p^ +j hl - p) 2 '^ (6) 

where 

(i) i 2 = 1 if a t = s 2 (h t -i) = s 2 (a t -i,b t -i), i 2 = otherwise, 

(ii) j 2 = liib t = sx(ht-i) = si(6 t -i, a*-i), j t 2 = otherwise. 

Now, by induction follows that p sl]S1 (at_i, & t -i) = p S2 , Sl { a t-ii°t~i) and from Si(/it-i) = s 2 (/t t -i) 
follows that ^ = if, = j 2 . □ 

Lemma 2. Given any pair of strategies s±, s 2 it follows that 

\ u sAK 2 , sl/ J\ < ^05) M 

where M = max{T, \ S\}. 

Proof. Observe that fixed t then 

E P si, s 2 {h t ) = 

ht&Ht 

since in the equilibrium path at time t the probability is p 2t+2 it follows that 

£ Psus 2 (ht) = l-P 2t+2 - 
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Therefore, and recalling that u{h l ) ^ M, 

\u 5 , P (K 2 , Sl/ht )\ = 



□ 

From previous lemma, we can conclude the next two lemmas: 
Lemma 3. Given two strategies s± and S2 

lim } U s Jsi,s 2 /h t ) = 0. 
h t eM£ 

Lemma 4. Given two strategies s\s 2 then 

Ump->iUs }P (s 2 , s 2 ) - U SjP (s 1 ,s 2 ) = sk l u s( h s2,s2/h k ) ~ u s(h si> s2/h k )]- 

Now, we are going to rewrite the equation @ considering at the same time the paths h and h. 
The reason to do that it will become more clear in subsection 17.11 

Remark 1. Observe that given a strategy s if ht 7^ ht it could hold that s{ht) 7^ s(ht). Also, 
given two strategies Si,s 2 it also could hold that kh(s\,s 2 ) 7^ kt(si,s 2 ). However, it follows that if 
kh(si,s 2 ) < fe^(si,s 2 ) then 

Ps llSl (h k ) =p SuSl (h k ) =p SliS2 (h k ) =p SuS2 (h k ) = 
Ps 2 ,si(h k ) =p S2tSl (h k ) =p S2tS2 (h k ) =p S2tS2 (h k ) 

Using previous remark, we define the set 1Z* S2 as the set 

n * Sl ,s 2 = i h G T^sx,s 2 '■ k h{s!,s 2 ) < kf t {si,s 2 )} 

and therefore the differences Us tP (s 2 , s 2 ) — Us )P (si, s 2 ) can be written in the following way (denoting 
k as fcft(s 1 ,s 2 )) 

Us, p (s 2 ,s 2 ) - U S)P (si,s 2 ) = 

^ 5 k p SuSl {h k )[Us^ p {si, si/h k ) - U StP {s 2 ,s 1 /h k ) + U 5 , p {si,si/h k ) - U SjP (s 2 ,si/h k )}. 

Now we are going to give a series of lemmas that relates equilibrium paths with seeds ht and ht] 
later, we also relate the payoff along those paths. The proofs of the first two next lemmas are 
obvious and left to the reader. 



1 - p 2 S 

p2 



<J*p 81 ,. a 



(l-<y)MjV(l-jr 
M[(l-«5)^-(l-5)^y'+ 2 ] 

M[l -p 2 



l-p 2 
(l-p 2 5) 



— ] 

1 -p 2 5 i 

M. 
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Lemma 5. Given two strategies s,s* and a path h t follows that 

K*,s/h t = h s,s*/h t ( 7 ) 
Now, we try to relates the payoffs. Given two strategies s, s* and a path we take 

61 = (1-<J) Yl fe 2 = (l-5) £ ^ 

j:ui(s*,s/h k )=R j:vJ (s* ,s/h k )=S 

63 = (1-5) J] «P, 64 = (1-5) J] 

j:vj(s*,s/h k )=T j:uHs*,s/h k )=P 

Observe that 61 + 62 + &3 + &4 = 1 and 

C/(s*, s) = 6 Xj R + b 2 S + 63T + biP. 
In the same way, for we define b\,b 2 ,b 3 , &4 

61 = (1-5) J] <P, 62 = (1-5) Yl P > 

j:ui(s*,s/h k )=R j:ui(s*,s/h k )=S 

63 = (1-5) Yl ^ ^4 = (1-5) ^ 

j:«i(s*,s/h fe )=T j:vj(s*,s/h k )=P 

Observe that 61 + 62 + ^3 + &4 = 1 • Now we define 

B 1 = h 1 + Si, £ 2 = 6 2 + &2, B 3 = b 3 + 63, S 4 = 64 + 64. 

Remark 2. T/ie a&ove numbers bj depend on 5 and i/ie infinite sums converge fixed 5. However, 
they could not converge as 5 goes to 1. 

Lemma 6. Given two strategies s,s* and a path h^, if 

Us(h s *, s/hk ) = hR + b 2 S + b 3 T + b 4 PS 

then 

U s(K,s*/h k ) = biR + b 2 T + b 3 S + b 4 P. 

Moreover, if 

U S {h a . ta/hk ) + U 5 (h s ^ s/hk ) = B 1 R + B 2 T + B 3 S + B 4 P, 

then 

Us(h SyS , /hk ) + U s (h at9 . /hk ) = B 1 R + B 2 S + B 3 T + B A P. 
Lemma 7. Given two strategies s,s* and a path follows that 

Us(h s ^ /hk ) + U 5 (h s , tS/hk )^2R. 

Lemma 8. For any Ao < 1 follows that there exists Ao < 1 such that if U$(h s s / ht ) = XqR then 

Us(h s>s/ht ) + U s (h ss/ht )^2X R. 

Moreover, for any X' < Ao then A' < Ao- In particular, 

U s (h s , s/ht ) + U s (h ss/ht )<2R. 



Proof. If U s (h s s/ht ) = b x R + b 2 S + b 3 T + b 4 P = X R then it follows that 

max{6 2 ,6 3 ,& 4 } > 3 ^° - 

In fact, if it is not the case, 

hR + b 2 S + b 3 T + b 4 P ^ hR = (1 - (b 2 + b 3 + 64)) > [1 - (1 - A )]i? = AnE, 
a contradiction. From equality ([7]) follows C7 (h g s /^ t ) = fti-R + + fr3<S + 64P so 

U 5 (h s , s / ht ) + Us(h S:S/ht ) = 2hR + (62 + 6 3 )(T + 5) + 26 4 P 
and from the fact that fei + 62 + ^3 + 6 4 = 1 follows that is equal to 

2R - [2b 2 (R - P) + (63 + bi){2R - (T + 5))] 

So taking 

£ = min{i? - P, R - ( r + S ) } 
which is positive, follows from inequality ([8]) that 

U S (h.,./h t ) + U S (h s>s/k ) < 2 ^ " 2^^P, 



(8) 



and taking 



the result follows. 



An = 1 



1-Xp R 
~ R 



□ 



From now one, given a pair of strategies s\ and s 2 (s 2 could be equal to s\) we use the following 
notations, 

Us, p (si, s 2 /h k ,h k ) := Us )P (si,s 2 /h k ) + Us, p (si, s 2 /h k ), 
U sA h s u s 2 /h k ;h k ) := U 5, P ( h si,S2/h k ) + U sA h s u s 2 /h k )- 

3 Replicator dynamics 

In this section we introduce the notion of replicator dynamics and we analyze the attractors. 
Given the payoff matrix 



A 



( an 
an 
\ am 



an 



' ' 11 



a\ n \ 



a-nn / 



Let A be the n— dimensional simplex 

A = {(a?i . . . x n ) E R n : x\ + • • • + x n = 1, Xj ^ 0, Vj}. 
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We consider the replicator dynamics X associated to the payoff matrix A on the n dimensional 
simplex given by the equations: 

Xj = Xj(x) := XjFj(x) = Xj{fj - f){x) (9) 

where 

n 

fj(x) = (Ax)j, f(x) = J2xifi(x), 

1=1 

where (AX)j denotes the j—th coordinate of the vector Ax. In other words, provided a payoff 
matrix A, the replicator equation is given by 

Xj = Xj[(Ax)j — x t Ax], j = 1, . . . , n 

where x t denotes the transpose vector. 

Using that 1 = x\ + X2 + • • • + x n we can write 

Fj = fj{x){xi+x 2 -\ \-x n )-f(x) = fj(x)(x 1 +x 2 -\ - ^2xifi(x) = ^2xi(fj - fi)(x). 

We denote with <p the associated flow: 

ip : R x A -)• A. 

Giving t G M. with (pt : A — > A we denote the t — time diffeomorphism. Observe that any vertex is 
a singularity of the replicator equation, therefore, any vertex is a fixed point of the flow. 

3.1 Affine coordinates for the replicator equation 

We consider an affine change of coordinates to define the dynamics in the positive quadrant of W 1 ^ 1 
instead of the simplex A. The affine change of coordinates is given by 

x\ = 1 — x h % = x j y J % 

and so, the replicator equation is defined as 

Xj = Fj(x)xj, j = 2, . . . , n 
where x = (x±,X2 • • • , x n ) with X{ ^ 0, x 2 + • • • + x n ^ 1 and 

F j(x) = (fj -/)(!- ^2xi,x 2 ,...,x n ). 

Observe that in these coordinates the point e± = (1,0,..., 0) corresponds to (0, . . . , 0) and in the 
new coordinates the simplex A is replaced by {(x 2 , ■ ■ ■ , x n ) : X{ ^ 0, Yll=2 x i ^^}- 
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We also can rewrite Fj in the following way: 

= E Ui-sm*i 



1^2 tyj,l22 

(/ J -/ 1 )(x)(i-E^)+ E Ui-sm^i 

1^2 1^3,1^2 

(/,-/i)(^)-E^-/i)(^+ E Ui-sm^i 

1^2 1^3,1^2 

(fj-h)(x)-(f j -h)(x)x j + E i(fj-fiKx)-(fj-hKx)]xi 

(fj-hm-(f j -h)(x)x j + e ih-fmxi 

(f j -f 1 )(x) + (f 1 -f j )(x)x j + E 

(/ j -/ 1 )(x) + E(/l-/«)(^- 



Z>2 



Denoting 



it follows that 



Z>2 



F j (x) = (f j -f 1 )(x) + R( 



x 



where 



(fj-fi)( x ) = ^2{a jk - ai k )x k = (aji - an)x! + ^2(a jk - ai k )x k 

fc^l k^2 

= (aji - a n )(l - E x i) + E ( a J' fc ~~ a ^) x ^ 

1^2 k^2 

= aji - an + E( a i fc ~~ a ' fc ~~ a 3 1 + a «i) x fe- 



Observe that if we take the matrix M £ Rt™- 1 )*^- 1 ) and the vector N G R n_1 such that 

Mjh = ajk — a\k + an — o-ji 

and 

Nj = aji — an 

then the replicator equation on affine coordinates is given by 

Xj = xj[(v + Mx)j - x l (v + Ma;)], j = 2, . . . ,n; 
where (v + Mx)j is the j — th coordinate of v + Mx. 
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Figure 1: Attracting fixed point. Basin of attraction. 

3.2 Attracting fixed points 

Given a point e and a positive constant e, B t {e) denotes the ball of radius e and center e. 

Definition 2. Attracting fixed point and local basin of attraction. Let e be a singular point 
of X (i.e.: X{e) = 0). It is said that e is an attractor if there exists an open neighborhood U of 
e such that for any x £ U follows that (ft(x) —> e. The global basin of attraction B s (e) is the set 
of points that its forward trajectories converges to e. Moreover, given e > we say that B t (e) is 
contained in the local basin of attraction of e if B € (e) is contained in global basin of attraction and 
any forward trajectory starting in B t {e) remains inside B e (e). This is denoted with B e (e) C Bf oc (e). 

For the sake of completeness, we give a folklore's sufficient condition for the vertex e\ to be an 
attractor. Before that, we need to calculate the derivative DX of the function X = {X\ . . . X n ) given 
by the replicator equation (see equation [9]). For that, for any I, we compute DXi = (§§^ • • • f^ 1 ) 
and observe that for k ^ I then = (d Xk fi — d Xk f)x^, and for k = I follows that Sj-^ = 
{d x Ji ~ d x J)xi + ft- f. 

Lemma 9. If e\ is a strict Nash equilibrium (i.e. an — aji > for any j ^ 1) then e\ is an 
attractor. Moreover, the eigenvalues of DX at e\ are given by {an — Oji}j>i- 

Proof. To prove the result, observe first that (the point e\ in the simplex) is a fixed point. To 
finish, observe that DqX is a diagonal matrix with {aji — anjj^i in the diagonal. Therefore, 
{oji — anjj^i are the eigenvalues which by hypothesis are all negative. 

□ 

3.3 Large Basin of attractions for fixed points 

The goal of the following theorem is to give sufficient conditions for a vertex to have a "large 
local basin of attraction", independent of the dimension of the space. In other words, provided a 
vertex e and a positive number K, the goal is to find sufficient condition for any payoff matrix A, 
independently of the dimension, such that the neighborhood Bn{e) is contained in the local basin 
of attraction of e. 



12 



A natural condition is to assume that the eigenvalues are "uniformly negative". But this criterion 
is not appropriate for the context of games, since the quantities aji — an even when negative could 
be arbitrary close to zero. However, we take advantage of the fact that the replicator equations are 
given by a special type of cubic polynomials, and we provide a sufficient condition for "large local 
basin of attraction" even for the case that the eigenvalues are close to zero. To do that, we need to 
introduce some other quantities. From now on we use the L 1 — norm 



Now, let us go back to the replicator equations and let us assume from now on that e is a strict 
Nash equilibrium, i.e. 

an — a-ji > 

for any j / 1. Recall as we define in the previous subsection the matrix M and N given by 

Nji = aji - an (13) 
Mij = aji - au + a n - aji (14) 
Mji = — aij + an - an. (15) 

Moreover, we assume that the vertex {e2 • • • e n } are ordered in such a way that 

an - an ^ an - aji, V2 < i < j. 

Theorem 1. Let A S W nxn (n arbitrary) such that aji < an. Let 



Mjj + Mjj 

M = max{ — J , 0). (16) 

i,j>i — iVj 

Then, 

The proof of the theorem is based on a crucial lemma about quadratic polynomials (see lemma 
[TO]) . So, first we recall a series of definitions and results involving quadric, we state the lemma, 
provide its proof and latter we prove theorem [TJ 

First recall that a quadratic polynomial Q is a function from M. n to R of the form Q(x) = 
Nx + x t Mx (where N is a vector, M is a square matrix and x t means the transpose of x). It is said 
that Q is positive- definite if x l Mx ^ for any x. It is said that Q is negative-definite if x l Mx ^ 
for any x. Now, associated to a quadratic polynomial Q we consider the set 

{x£R n : Q{x) = 0} 

which is a smooth submanifold of codimension one. Observe that Q(0) =0. If Q is either positive- 
definite or negative-definite then {Q(x) = 0} is an ellipsoid, in particular, it is a connected compact 
set and {x E M. n : Q(x) ^ 0} is a convex set (see first two cases in figure [2]). If Q is neither 
positive-definite nor negative-definite then {Q(x) = 0} is a hyperboloid, and in particular, it is not 
a bounded set. However, it could be connected or not (see third case of figure [2]). 



13 



Figure 2: Q positive-definite, negative-definite and neither. 



Lemma 10. Let Q : R n — > R given by 

Q(x) = Nx + x*Mx 

ura'i/t x G M n ; ./V G M n and M G M nxn . Lei us assume that iVj < /or any i and for any j > i ; 
| JVil ^ | iVj | . Let 

M = max{ — J , 0). 

i,j>i — iVj 

Then, the set = {x G M™ : X{ ^ 0, X^=i < IT"} * s contained in {x : Q(x) < 0}. In particular, 
if Mq = then is treated as oo and this means that {x G M n : X{ ^ 0} C {x : Q(x) ^ 0}. 

Proof. For any t> G M n such that f j ^ and ^ t>j = 1, we consider the following one dimensional 
quadratic polynomial, Q v : M — > M. given by 

Q v (s) := Q{sv) = sNv + s 2 v t Mv. 

To prove the thesis of the lemma, we claim that is enough to show that 

"for any positive vector v with norm equal to 1, if < s < — — thenQ v (s) < 0"; (17) 

in fact, to prove that claim, we can argue by contradiction: if there is a point xq G different 

M 

than zero (i.e.: < \xq\ < -p^) such that Q{xq) = 0, then taking v = t^t and s = \xq\ follows that 

Q v (s) = Nxq + XqMxo = 0, but \v | = 1, s < jg-, a contradiction. 

Now we proceed to show (|17p . Observe that the roots of Q v {s) are given by s = and 

-Nv 



v t Mv 

Observe that 

-Nv = J2(~ N i>i > 0. 

If v l Mv < then it follows that Q v is a one dimensional quadratic polynomial with negative 
quadratic term and two non-positive roots, so for any s > holds that Q v (s) < and therefore 
proving the claim in this case. So, it remains to consider the case that v t Mv > 0. In this case, 
since Q v is a one dimensional quadratic polynomial with positive quadratic term (f*Mv), therefore 
for any s between both roots (0, IIjJL ) follows that Q < so to finish we have to prove that 

-Nv 1 , . 

-JJ77. > — ■ ( 18 ) 



v l Mv M 
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Using that X^j>i v j ^ 1 observe 

v t Mv = y^VjVjMjj 

ij 

= ^K 2 Mii + J] ViVj(Mij + Mfi)] 

< Y^(-Ni)M + v l v j (-N i )M ] = 

i j>i 
i j^i 
i 

= Mo(-Nv). 

Therefore, (|18p holds and so proving (|17D . □ 



Now we provide the proof of theorem [TJ 
Proof of theorem^ We consider the afhne change of coordinates: x\ = 1 — Y2j>2 x j' x j = x i->3 = 
2,...,n introduced before. Let X = (X2,---,X n ) the vector field in these coordinates, where 
Xj = XjFj(x). For any k < 1 we denote 

A fc := {x : ^ Xi < fc}, <9A fc = {x : ^ x f = k}. 

i^2 i^2 

We want to show that for any initial condition x in the region A^_ follows that the map 

M 

t ->• = ^ Xk(t) 

i^2 

is a strict decreasing function and so the trajectories remains inside A^_ and since it can not escape 
A it follows that x(t) — > and therefore the trajectory converge to (0, . . . , 0). To do that, we prove 

x < 0. 

Therefore, we have to show 

Q(x) := x = J2 Xj =Y xjFjix) <0. (19) 

Recall that Fj = (fj — fi)(x) + R(x) where R(x) = Ylii^2 (/l ~~ fl)( x ) x l ( see equations (fTU|) and 
pip ). Therefore, 

Q( x ) = ^(fj-fiMxj + ^R^Xj 

J>2 j>2 



Since Xw>2 = ^ (with fe < 1) follows that 



Q( x ) = ^2(f3-fi)(x) Xj + R(x)k. 
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Recalling the expression of R we get that 

Q(x) = (l-k)J2(fj-h)(^r 

So, to prove inequality (|19p is enough to show that 

Q(x) = (l-k)^2x j (f j -f 1 )(x)<0 VxG A fc , ^<^-- 
j 

First we rewrite Q. Observe that 

(fj-fi)(x) = ^2 (aji - au)xi = 

i 

= Qji - an + y^( a ji ~ a l« + a H ~~ a j'l) X i- 



If we note the vector 
and the matrix 
Therefore, 



N := (oji - an)j 
M := (Mij) = aji - an + an - a 3 -i. 
g(x) = Nx + i'Mx. 



So we have to find the region given by {x : Q(x) = 0}. To deal with it, we apply lemma [TUl and we 
use equation (|16p and the theorem is concluded. 

□ 

Remark 3. Observe that in the theorem [ZOl it only matters to compare an — an with the entries 
Mij + Mji that are positive. 

Remark 4. If we apply the proof of lemma[IU to the particular case that v = ej, we are considering 
the map 

Q v (s) = s[aji — an + (o,jj — aij + an — aji) s] 
and Q(s) =0 if and only if s = or 

Pij (20) 



an ~ aji + djj - aij \ + 



3 jj -°i j 



au—aji 

and so 

Q(s) < 0, VO < s < pij. 

In particular, if we apply this to theorem UJK it follows that the whole segment [0,pij) is in the basin 
of attraction of e\. In particular, observe that pu = (1 — pu, . . . ,pu . . .), is the unique fixed point 
of the replicator dynamics in the interior of the one dimensional simplex that contains ei, e,-. 

Remark 5. Observe that the basin of attraction could be much larger than the region given by the 
previous theorem. It may be the case that better linear upper bounds for the quadratics map Fj could 
provide better estimates for the size of the basin of attraction. 
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3.4 Comparing strategies by pairs is not enough 

In this section we show that to guarantee that a strategy has a uniformly large basin of attraction 
is not enough to compare it with every other single strategy one at the time. In other words is not 
enough to bound by below the basin of attraction of only considering populations of two strategies. 
More precisely, we provide a set of conditions for a population of three strategies where only one 
is an attractor (and therefore its basin is large in each one dimensional simplex) but it has a small 
local basin. 

We consider a replicator dynamics in dimension two and we write the equation in affine coordi- 
nates {(x\,X2) '■ ^ X2 ^ 1, x 3 ^ 1, X2 + x 3 ^ 1}. Given A > and close to zero, we consider 
the almost horizontal and vertical lines given by 

H\{x 2 ) = {x 2 , A(l - x 2 )), V\(x 3 ) = (A(l - x 3 ),x 3 ). 

Theorem 2. Given A > close to zero and a > 0, there exist A G M 3x3 such that < ajj < a, 
satisfying that 

(i) (0,0) is an attractor and the horizontal line (xi,0),0 ^ x\ < 1 and vertical line (0,^2), ^ 
X2 < 1 are contained in the basin of attraction of (0,0); 

(ii) (1,0) and (0,1) are resellers; 

(Hi) there is a point p = (pi,P2) with pi + P2 = 1 which is an attractor; 

(iv) the region bounded by H\, V\ and x\ + X2 = 1 is contained in the basin of attraction of p. 

Proof. To prove the result, we choose A G ]R 3x3 such that for any (x2,x 3 ) G H\ and (x2,x 3 ) G V\ 
follows that X(x2,x 3 ) points towards the region bounded by H\,V\ and x\ + x 2 = 1. For that, it 
is enough to show that 

*(**(.>» _ A(l - * 2 )F 3 (* (* 2 )) > 1 > „ f A < ^ < 



X 2 (H x (x 2 )) |x 2 F 2 (ff(x 2 ))| 4' *" 1-A 



and 



X2(Vx(x 3 )) X(l-x 3 )F 2 (V(x 3 )) 1 A 

X^O^)) = |*3F 3 (n*3))| 4' > ° fol l3A < * 3 < X > (22) 

where (j^, j^) is the intersection point of H\ and V\. Recall the definition of N G R 2 , M G M 2x2 
that induce the replicator dynamics in affine coordinates. Given A we assume that 

(i) iV 2 = iV3, 

Mm. — M33 — I 

\ L1 J N 3 ~ N 3 A ' 

To get that, and recalling the relation between the coordinates of M and A (recall identities I13p. 
we choose the matrix A such that 



(\\ Q33— Q 13 — Q 0-22— 112 — Q. 
^ ^3 ~~ ' W2 ~ ' 

(ii) a 32 > a 22 , a 23 > a 33 and = 023=33 = 1 _ 2 . 
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With this assumption, now we prove that inequality (|2ip is satisfied: Let us denote x := x 2 and we 
first calculate F^(x, X(x — 1)) and F 2 (x, X(x — 1)), 

F 3 (x, A(l - x)) = N 3 + M 32 x + M 33 A(1 - x) - 

[x(N 2 + M 22 x + M 23 A(1 - x)) + A(l - x){N 3 + M 32 x + M 33 A(1 - x))} 



so, 



so, 



r .JV 2 M 22 M 2 3 w , s w w M 32 M 33w 

+ + ^fA(l - x)) + A(l - x)(l + -j^x + ^A(l - x))] 

= 1 + -x + 2A(l-x)- 
A 

[x(l + 2x + -A(l - x)) + A(l - x)(l + -x + 2A(1 - x))] 
A A 

= 1 + 2A + (- - 2A)x - [2A 2 + A + (3 - A - 4A 2 )x + 2A 2 x 2 ] 
A 

= 1 + A - 2A 2 + (- - A + 4A 2 - 3)x - 2A 2 x 2 , 
A 

F 2 (x, A(l - x)) = N 2 + M 22 x + M 23 A(1 - x) - 

[x(N 2 + M 22 x + M 23 A(1 - x)) + A(l - x)(N 3 + M 32 x + M 33 A(1 - x))] 



F 2 (x,A(l-x)) M 22 M 23 

77 = 1 + —Z7- x + All — X) — 

N 2 N 2 N 2 y ' 

M 22 M 23 M 32 M 33 

+ 1^ X + " X)) + A(1 " X)(1 + ~N2 X + A^ A(1 " X))] 

= 1 + 2x + ^A(l -x) - 
A 

[x(l + 2x + -A(l - x)) + A(l - x)(l + -x + 2A(1 - x))] 
A A 

= 2 + x - [x + 2x 2 + (1 - x)[l + A + 2A 2 + (1 - 2A 2 )x]] 
= (1 - x)[2(l + x) - [1 + A + 2A 2 + (1 - 2A 2 )x]] 
= (1 - x)[l - A - 2A 2 + (1 + 2A 2 )x]. 

Therefore, on one hand observe that 1 + A - 2A 2 + (± - A + 4A 2 - 3)x - 2A 2 x 2 is a quadratic 
polynomial with negative leading term that is positive at 1 and (provided that |A| is small) so 
is positive for < x < 1, on the other hand (1 — x)[l — A — 2A 2 + (1 + 2A 2 )x] is positive in the 
same range, so 

A(x- l)F 3 (x,A(x- 1)) _ A[l + A - 2A 2 + ({ - A + 4A 2 - 3)x - 2A 2 x 2 ] _ 
|xF 2 (x,A(x - 1))| ~ x[l- A - 2A 2 + (1 + 2A 2 )x] ' 

since the minimum of the numerator is attained at getting a value close to 1 and the maximum 
of the denominator is attained at 1 getting a value close to 2, follows that in the range < x < 1 
holds 

A(x-l)F 3 (x,A(x-l)) 1 



|xF 2 (x,A(x-l))| " 3 : 
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(0.1) 




(0,0) 



(1.0) 



Figure 3: Comparing strategies by pairs is not enough. 



and therefore the inequality (|2ip is proved. The proof of inequality (|22p is similar and left for the 



Remark 6. Observe that under the hypothesis of theorem^ the point (j^j, j+i) ^ s n °t ^ n ^ e basin 
of attraction of e\. 

Remark 7. It is natural to wonder if the conditions of theorem Q] are necessary? More pre- 
cisely, theorem^ provides a lower bound in the size of the basin of attraction related to information 
on the payoff matrix. We could wonder if the lower bound also works as an upper bound: is it 
true that if is small then the basin of attraction is small? Previous shows that under certain 
conditions, this is the case. 

4 Uniformly large basin of attraction 

In the rest of the paper we study the replicator dynamics when the matrix of payoffs is given by a 
finite set of strategies S = {s\, . . . , s n } from an infinitely repeated prisoners' dilemma game with 
discount factor S and error probability 1 — p. It is well known, that any strict subgame perfect is 
an attractor in any population containing it. In this case, with Bi oc (s,5,p,S) we denote the local 
basin of attraction of s in any set of strategies S and identifying s with s%. Related to that we give 
the following definition: 

Definition 3. We say that a strategy s has a uniformly large basin if there is Kq verifying that for 
any finite set of strategies S containing s and any 5 and p close to one, it holds that 



where n = cardinal{S) . 

One particular case of previous definition is when S has only one strategy different than s. In 
this case, and based on remark d] we can obtain the following remark: 



reader. 



□ 



{(xi, ...,x n ): x 2 H hx n < K } C B t 



S) 
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Lemma 11. If s has a uniformly large basin then there exists Cq such that for any strategy s* and 
for any p,S large (independently of s*) follows that 

Us, p (s,s) - U s , p (s*,s) 

In particular, 

s^ip^i Us,p{s, s) - Us :P {s*, s) 

The goal of this paper is to understand which characteristics of strategies lead them to have 
uniformly large basin of attraction. We show first that a strategy that is commonly used in the 
literature, grim, does not have a uniformly large basin of attraction. Then, we show that is due 
to the fact that grim never forgives a defection. As a positive results we show that another well 
known strategy, win-stay- lose-shift, does have a uniformly large basin of attraction under certain 
conditions. 



5 The importance of forgiveness 

In this section we show the importance of forgiveness for the evolutionary robustness of strategies. 
First we prove that neither the strategy Grim nor Always Defect has a uniformly large basin of 
attraction. Recall that Grim is the strategy that cooperates in the first period and then cooperates 
if there has been no defection before. We provide the proof for Grim and we observe that this proof 
is obviously adapted for Always Defect. T prove that Grim (g from now on) does not have a large 
uniformly basin of attraction, we are going to find a strategy s such that when the population with 
g and s is considered, the basin of attraction of g is arbitrary small provided that 5 and p are close 
to 1. In fact, we use the equation tj20jl to determine the boundary point p 9:S = — v ( s }- } _ Ug (g s) of 

~^ U S,p(S:3)-Us yP (s,g) 

the basin of attraction of g (the smaller p 9tS is, the smaller the basin of attraction of g is). Relatedly, 
Myerson [MJ proved that whenever the strategy Always Defect is compared with Grim (without 
tremble), its basin of attraction collapses as the discount factor converges to one. 

Theorem 3. Grim does not have a uniformly large basin of attraction. More precisely, there exists 
a strategy s such that for any population S = {s, g} and e > small, there exist Po,5q such that for 
any p > po, S > 5o, the size of the basin of attraction of grim is smaller than e. 

Proof. We consider the strategy s that behaves like g but forgives defections in the first period 
(t = 0). We need to show that for any e > small, there exist po, 5q such that for any p > po, 5 > Sq, 
follows that 

1 

^ , Us,p(s,s)-Us, P (g,s) e ' 
Us,p(g,g)-U s ,p(s,g) 

From the definition of s, for any h verifying that h° ^ (D, D) and any t it follows that 

Pg,g(ht) = Ps,g(h t ) = Pg,s(ht) = Ps,s(h t ). 

Therefore, 

Us, p (s,s/(C,C)) = U s>p (s,g/(C,C)) = U s>p (g,g/(C,C)) = U Sj> (g,s/(C,C)), 
U SiP (s,s/(D,C)) = U s>p (s,g/(D,C)) = U SlP (g, g/(D,C)) = U S}P (g,s/(D,C)), 
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U s , p (s,s/(C,D)) = U s , p (s,g/(C,D)) = U S:P (g,g/(C, D)) = U s , p (g, s/(C, D)), 

so 

U Sj ,{s,8) - U 5tP (g,s) = U s , p (s,s/(D,D))p s , s (D,D) - U s , p (g, s/(D, D)) Pg>s (D, D), 

U s , p (g,g) - U 5 , p (s,g) = U 5 , p (g,g/(D,D))p g , g (D,D) - U s<p (s, g/(D, D)) Ps>g (D, D). 

Recalling that s after (D, D) behaves as g and g after (D, D) behaves as the strategy always defect 
(denoted as a) and p s , s (D, D) = p s , g (D, D) = p g , s (D, D) = p g , g {D, D) = (1 - p) 2 , then 

Us,p(s,s) - U SjP {g,s) = (1 -p) 2 S[U S:P (g,g) - U S)P {a,g)\, 

Us, P {g,g) - U StP {s,g) = (1 -p) 2 S[U SjP (a,a) - U SjP (g,a)]. 

Therefore, it remains to calculate the payoffs involving o and g. Also observe that for any path h 
if we take k as the first non-negative integer such that h k ^ (C, C) then for any t > k p Sl)S2 (ht) = 
Ps 1 ,s 2 {hk)Pa,a{<7 k (h)t~k) where si and S2 is either g or a and a k (h) is a history path that verifies 
a k (h)j = h j+k . 
Therefore 

Us, P (9,g/ h k) = U 5tP / hk (a,g) = U s , p (g,a/h k ) = U SjP / hk (a,a). 
So, noting with (C, C) 4 a path of t consecutive simultaneous cooperation and 

L = E b t PaAht)u{h t ) = T ^- § [{l-p) 2 R+{S + T){l-p)p + p 2 P\, 

t^0,h t 

follows that 

Us, P {g,g) - U 5 , p (a,g) = 

(i - tup, c)\p g , 9 m cf) - Pa , 9 m cm + 



Z>) + SL]\ Pg , g ((C,C) t (C,D) -p a , g {{C,C)\C,D))] + 

<**[«(£>, C) + <JL][p 9>9 ((C, C)'(A C)) - Pa , g ((C, Cf(D, C))] + 
£ 5*[u(D, D) + Mfc ((C, C)'(A D)) - p a J(C, Cf(D, D))}} = 
(1 - ^ l R[p 2t - p\l - pf] + Y,^iS + SL] [p 2t p(l - p) - p\l - p)\\ - p) 2 ] + 

#[T + *£] [P 2t (l - P)P - P*(l - + E ^ t P + 5L 1 ^'U " ^ " ^ " " P)f]>- 



Therefore 

U8#{9,g) ~ U s , P (a,g) = (1 - <5)GA(5,p) 

where 

[r + tt][ (1 - p 'f f 2 i + p + anC-f) 2 I 1 "* 



1 — p 2 5 1— p(l — p)5 l — p 2 5 1 — p{\ — p)5 
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and we write 
where 



GA(5,p) = GA°(5,p) + GA\5, P ) 



G ^ (M " - 1 _ p( i _ p)5 ] + S ^T^pH ~l- P (l- P )5 ] + 

r r (! ~P)P _ P 2 1 pr (! ~P) 2 _ (1 ~ P)P 1 ^ 

[ l-p 2 (5 l-p(l-p)<5 J [ l-p 2 5 l-p(l-p)<5 J 
[Rp 2 + (S + T)p(l - p) + P(l - p) 2 



1 — p 2 5 1 — p(l — p)<5 ' 



^ rF P(l ~P) (l~P) 2 1 , 



p 2 5 1 — p(l — p)5 

dL n — zjt ~ 1 — it; — 37J + 



l — p 2 5 1 — p{\ — p)5 l—p 2 5 1 — p{l — p)5 

An !~P 2 _ 1- (1-p)p 1 
L l_p2$ l-p(l-p)5 J " 

Observe that when p, S — > 1 then 

V + (S + T) P (i- rt + P(i-p)^«, Y^^-i. rqfe^ ^ 1 

and recalling that (1 — 5)L = P = (1 - p) 2 i? + (S 1 + T) (1 - p)p + p 2 P then for 5, p large follows that 

(1 - S)GA°(i,p) > |(Y^ (23) 

(1- { ) G ^ ( M^<^. (24) 

In the same way 

Us, p {a,a) - Us, p {g,a) = 

(1 - 6){J2 <J*«(C, C) [p a , a ((C, C)«) - p fl , a ((C, C)*)] + 



^^KCD) + < 5L][p a , a ((C,C)*(C, J D) -^((CC/ICD))] + 

+ ,5L][p aia ((C,C) t (AC)) -^.((CCj'p.C))] + 
^ $*[«(£>, D) + 5L]\p a , a ((C, C)\D, £>)) - p ff , a ((C, C)\D, D))}} = 

(1 - S'-'RKl - P ) 2t - p\l - pf] + £ <J* [S + 5L \ K 1 " P) 2 ^ 1 -P)~ " P) VI + 

2 <5'[T + SL] [(1 - p) 2t (l - p)p - p\l - p)\l - p) 2 ] + 

Y, #[p + ^][(i - p) v - p*(i - p)*(i - p)p]}- 
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Therefore 

U s , p {a,a) - Us, p (g,a) = (1 - S)AG(6,p) 

where 

AG(M " - (1 - p) 2 5 ~ 1 - p(l - p)5 ] + [5 + - (l - p) 2 5 " 1 - p(l - p)J ] + 

[r+fL][ (1 ,~ p) * - {l : p) \ x ] + [p+mv- " 2 {1 ~ p)p 



l-(l-p) 2 5 l-p(l-p)5 i L JL l-(l-p) 2 <5 l-p{l-p)5 i 

and we write 



AG(5,p) = AG°(6,p) + AG 1 {8,p) 



where 

-<0 



^G U (<5, P ) - R[ 1 ^ (l ^ p)26 y^Ci - p) s ] + S[ i- { (i- P p)25 i- p (i-p)5 ] + 

P 2 (1 ~ p)p i 

{l-p) 2 5 l-p(l-p)<5 J 

AG\5,p) = SL[- 



[ ( l - 


■P) 2 




p(l-p) 




— p) 2 5 


1 


-p(l-p)5 l 


■ (1- 


P)P 




(1-P) 2 , 


■1-(1 


— p) 2 5 


1 


— p(l — p)<5 


p(l- 


-P) 




P 2 


1-(1 


-p) 2 5 


1 


-p(l-p)<5 J 


(1- 


P)P 




(1-P) 2 , 


1-(1 


-p) 2 5 


1 


-p(l-p)<5 J 


2p(l 


-P) 




1 -p 


1-(1 


-p) 2 5 


1 


-p(l-p)<5 J 


2p(l 


-P) 




1 -p j 



1-(1- 


p) 2 <5 


l-p(l- 


p)5- 


p 2 




p 2 




1-(1- 


p) 2 <5 


l-p(l- 


p)<5- 




P 2 (l- 


-p) 





+ <5L 

^1 - (1 -p) 2 5 ~ l-p{l-p)5 [ + SL \\ - (1 - p) 2 5){\ - p(l - p)<5) J " 

» f1 _ u 2p 1 -p pH 

1 (l-p) 2 5 l-p(l-p)<5 (l-(l-p) 2 <5)(l-p(l-p)(5) J 

Observe that when p, S — > 1 then 

AG°(5,p) ->• AG°(1, 1) = P - S, 
2p 1 — p p 2 (5 



l-(l-p) 2 (5 l-p(l-p)<5 (1 - (1 -p) 2 (5)(l -p(l -p)5) 
and recalling that (1 - S)L = P = (1 -p) 2 R+ (S + T)(l -p)p + p 2 P then for S,p large follows that 

(I- 5)AG°(5,p) <2(l-(J)(P-5) (25) 

(1 - a^G 1 ^) <4(1- p)P. (26) 
Recall now that the size of the basin of attraction of a is given by 

(l-5)AG(«5,p) 
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Observe that for any e > for p, 5 large then from inequalities (I23p and (|25p 

(1 - 6)AG°(5,p) ^ e(l - 5)GA°{5,p) 



and from inequalities (I24|) 



and (l26]l 



(1 



<5)AG e(l 



c^GA 1 ^) 



therefore, for p, 5 large 



i 



e 




1 + e 



and so the theorem is concluded. 



□ 



Theorem [3] shows that the well known strategy grim does not have a uniformly large basin of 
attraction given that after a defection it behaves like always defect, which does not a uniformly 
large basin of attraction either. In an world with trembles unforgivingness is evolutionary costly. 
We formalize next the idea of unforgivingness and and provide a general results regarding the basin 
of attraction of unforgiving strategies. 

Definition 4. We say that a strategy s is unforgiving if there exists a history ht that s{ht+ T /ht) = D 
for all h t+T with r = 0, 1, 2.... 

Theorem 4. Unforgiving strategies do not have a uniformly large basin of attraction. 

The proof is similar to the proof of theorem [3] with the difference that the first point of divergence 
may not be t = 1. 

6 Efficiency and size of basin of attraction 

In the present section we study the relationship between efficiency of a strategy and the size of its 
basin of attraction. Roughly speaking, full efficiency means that strategies cooperate with itself. 
Given a history ht, and a pair of strategies s,s* we define 



In next subsection we prove that strategies having a uniformly large basin of attraction are 
efficient provided that the strategies are symmetric. In subsection 16.21 we discuss the case of non- 
symmetric strategy provided some form of weak efficiency. 

6.1 The symmetric case 

Definition 6. We say that a strategy s is symmetric if for any finite path ht it follows that 



Theorem 5. If s has a uniform large basin of attraction and is symmetric, then it is efficient. 

The previous result established efficiency if the probability of mistake is much smaller than 1 — 5. 
An easy corollary is the following: 



U (s, s/h t ) = lim lim Us :P (s, s/ht). 



Definition 5. We say that a strategy s is efficient if for any finite path ht follows that 



U{s,s/h t ) = R. 
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Corollary 1. If s has uniform large basin of attraction and is symmetric, then for any Rq < R 
there exists 5q := Sq(s) such that for any 5 > 5q there exists po(S) verifying that if 5 > 5o,p > po(S) 
then 

U s>p (s,s/ht) > i? 

for any history ht- 

Here it is important to compare the statement of theorem [3] with theorem [5] and corollary [TJ 
First, observe that the conclusion of theorem [3] is obtained for any 5 > 5q and any p > po; instead, 
in corollary [JJ is for d > 5q but p > p(<5)with p(5) strongly depending of 5. Second, a weaker version 
of theorem [3] can be concluded from corollary [TJ 

Lemma 12. If s has a uniformly large basin of attraction, then there exists Cq such that for any 
s* and ht follows that 

Us, p {s*,s*/h t )-U s ^s,s*/h t ) + U 5 ^(s*,s*/h t )-U 5 ^{s,s*/h t ) n 
lim hm — < Co- 

5^i P ^i u s , p (s, s/h t ) - U s , p (s*,s/h t ) + U SiP (s, s/h t ) - U s , p (s*, s/h t ) 

Proof. It follows immediately from lemma [TT1 considering a strategy s* such that the first deviation 
from s occurs at ht (and obviously also at ht). 

□ 

Proof of theorem 0- Let us assume that there exists a path ht and Ao < 1 such that 

U(s,s/h t ) = XoR 

and s is a sub game perfect. We start assuming that ht is not symmetric. Then we show how to 
deal with the symmetric case using the asymmetric one. 
From the fact that s is symmetric, then follows that 

U(s, s/ht) = U(s, s/ht) 

and therefore 

U(s, s/h) + U(s, s/ht) = 2\ R. 

Moreover, since U(s,s/ht) < R, we can assume that s(ht) = D. We are going to build a strategy 
s* such that 

(i) U(s*,s*/h t ) = U(s*,s*/h t )=R, 

(ii) s* acts like s after meeting s at ht and ht- 

To build that strategy s*, first we take s* such that s*(ht) = s*(ht) = C and then we consider all 
the paths that follows after ht, ht for the pairs s, s; s* , s; s, s*; s*, s*: 

(i) h t (D,D),h t (D,D) for s,s 

(ii) ht(C,D), ht(C,D) for s*,s 

(iii) h t (D,C), h t (D,C) for s,s* 

(iv) h t (C,C), h t (C,C) for s*,s*. 
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Observe that the paths involving ht are all different and the same holds for the paths involving fit- 
Now we request that s* after ht(C, C) and ht(C, C) plays C for ever, so 

K*, s * /ht = (C, C) . . . (C, C) . . . , h^ /ht = (C, C) . . . (C, C) . . . , 

and so 

U(s*,s*/h t ) = U(s*,s*/h t ) = R. 

We also request that 

s*(h t (C,D)) = s(h t (C,D)), s*(h t (C,D)) = s(ht(C,D)), 
and observe that both requirement can be satisfied simultaneously and inductively we get that 

h s *,s/ht(C,D) = h,s/ht(C,D), h s *,s/ht{C,D) = h s,s/h t (C,D)- 

From the fact that s is symmetric, it follows that each entry of /is*,s//t t (C.r>) = h-s,s/h t (C,D) an d 
h gSt s /h t toD) = s/h t (C D) ^ S ^0 or [PiD") and recalling equality [3 follows that 

U(s*,s/h) + U(s*,s/h) = U{s, s*/h) + C/(s, s*//i). 

Since, s is a sub game perfect (otherwise it would not have a uniform large basin of attraction) then 
U(s*,s/h t ) + U{s*,s/h t ) < 2X R and therefore U(s,s*/h t ) + U(s,s*/h t ) < 2X R; by remark (fT2l) 
follows that if we denote U(s*,s/h t ) + U(s*,s/h t ) = 2XiR, then 

' " AL < Co, (27) 



Ao — Ai 

and taking a positive constant C\ < 1 — Ao < 1 — Ai it follows that Ai satisfies inequality 

Ao — Ai 

Therefore, it follows that there exists 7 > such that 

Ai < A - 7. 

Now, we consider the path ht(C,D) and we denote it as ht 2 and as before we construct a new 
strategy that satisfies the same type of properties as the one satisfied by s* respect to s but on 
the path ht 2 instead on the path ht- Inductively, we construct a sequences of paths ht it strategies 
s* and constants A-; such that 

U(s*,s/h ti ) = X i R (29) 
and they satisfy the following equation equivalent to (I27p 

T— < C , (30) 
Aj — Aj-i-l 

and since Aj + i < Ai then also satisfy an equation equivalent to (I28p 

I C \ < Co- (31) 

Aj — Aj+i 



26 



and therefore 

Aj+i < Aj - i-y 

but this implies that A.; — > — oo and so U(s*,s/ht i ) — > — oo, a contradiction because utilities along 
equilibrium are bounded by P. 

To finish, we have to deal with the case that ht is symmetric and U(s,s/h t ) < R. Recall that 
we can assume that s(ht) = s{ht) = D. Now, let us consider the sequel path ht(C,D). We claim 
that if U(s, s/ht) < R then 

U(s,s/h t (C,D)) < R. 

In fact, we can consider the strategy s* such that only differs on ht and after that plays the same as s 
plays. Since s is a sub game perfect (otherwise it would not have a uniform large basin of attraction), 
it follows that U$ tP (s, s/h t ) ^ U$, p (s*,s/ht) therefore, U(s,s/ht) = lim^i lim p _>i U$ tP (s, s/ht) 
lim^^i linip-^i Us tP (s*, s/ht), but since 

lim lim U s p (s*, s/ht) = lim lim Us v (s,s/ht(C,D)) = U(s,s/ht(C,D)) 
<5->lp->l <5^1p-s>l 

the claim follows. 

Observe that the new path ht(C,D) now is not symmetric and since U(s,s/ht(C,D)) < R, to 
conclude the proof of theorem [5] we argue as above. 

□ 

It remains to be shown that there exists strategies with uniformly large basins of attraction. 
To do that we must first develop some simple way of calculating payoffs under the presence of 
trembles. This calculations will help us prove that there exist strategies with uniformly large basins 
of attractions. 

6.2 Non-symmetric strategies 

We are going to provide a series of results about weak forms of efficiency for strategies having a 
uniform large basin of attraction without assuming that the strategies are symmetric. The first 
ones actually only compare pair of strategies; in other words, it is only used that the strategies has 
a uniform large basin of attraction in populations with two strategies (see propositions Q] and [2] and 
theorem [6]); moreover, theorem [6] gives a lower estimate of the size of the basin related to a quantity 
that measure the non-symmetry of a strategy. 

In theorems [7] and [8] we show that if there is a history such that a strategy is not fully efficient 
for any subsequent path, then it can not have a uniform large basin of attraction. The proof of 
both theorems are based on theorem [2] where we analyzed the dynamics of a population of three 
strategies. 

Proposition 1. If s has a uniformly large basin of attraction, then there exists e > such that for 
any ht follows that 

U{s,s/h t ) >P + e. 

Proof. Choosing s* such that s*(ht) ^ s(ht) and for any hk containing ht(s* (ht), s(ht)) then 
s (h-k) = D follows that U(s*,s) ^ P and U(s,s*) ^ P. Since also we can chose s* such that 
^s* s*Jh t {{s*(h t ) s*(h t )) ^ s a P a th °f f uu cooperation, then by lemma [TT] the conclusion of the proposi- 
tion follows. □ 

Observe that previous result is stronger that theorem 0] (provided that p is much closer to one 
than 5) since here it is shown that strategies with uniformly large basin of attraction have a payoff 
uniformly away from zero. Next result goes in the same direction but relating payoff with the size 
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of the basin of attraction; actually, a lower bound in the basin of attraction provides a lower bound 
on the payoff that a strategies get when plays with itself. 

Proposition 2. If s has a uniformly large basin of attraction and for any p and 5 large follows that 
there exists k verifying Bk(s) C B s (s), then 

U(s,s/h t ) > P +{R- P)k. 

Proof. The proof is similar to the proof of proposition Q] and using remark U] that estimate the size 
of the basin of attraction when only two strategies are involved. 

□ 

Next definition is related to the asymmetry of a strategy. In few words, it measures how frequent 
is s(hf) ^ s(hf) along a path. 

Definition 7. Given a strategy s, it is said that s is c— asymmetric if for any h% holds 

j:ui(s,s/h t )=T j':ui'(s,s/h t )=S 

and there are paths such that ^2j. u j^ s s /h t )=T ^ Sf-ui' (s s/h t )=S ^ * s ar °ttrary close to c. In par- 
ticular, if s is 0— asymmetric, then it follows that it is symmetric. 

The next two lemmas relates the payoff of s with s starting at ht and starting at ht- 
Lemma 13. Given a strategy s and a history ht it follows that 

U s (s, s/ht) = U 5 (s, s/h t ) + {x-y)(T- S), 
Where X = Ej:ui(s,s/fH)=S & and V = Ef: U i'( s ,s/h t )=T Sj - 

Proof. If U s (s, s/h t ) = aR + xS + yT + bP where a = J2j: U j(s,s/h t )=R 63 and b = T,f:vj'(s, s /h t )=P 53 
then U s (s, s/ht) = aR+xT+yS+bP = U s {s, s/h t )+xT+yS-xS-yT = U s (s, s/h t ) + (x-y){T-S). 

□ 

Lemma 14. Given a strategy s and a path ht it follows that if c = x + y then Us(s,s/ht) ^ 
U 5 (s, s/h) + c{T - S) if c < ^zM^M an d u 5 (s, s/ht) ^ -U 5 {s, s/ht) + 2R + c(T + S - 2R) 
otherwise. 

Proof. From lemma [13] follow that Us(s,s/ht) = U$(s, s/ht) + (2cc — c)(T — S). To conclude, 
observe that under the restriction x + y = c, a + c + b = 1, a,b,x,y are in [0, 1] and U$(s, s/ht) = 
aR + xS + yT + bP, the maximum of 2x — c is equal to c if c < — u ^'^ ht ^ anc l i s equal to 
2 R-u^s,s/h t ) + c t+s^r otherwise- 

□ 

Theorem 6. If s has a uniformly large basin of attraction in populations with two strategies and is 
c— asymmetric, then for any ht follows that 

U(s,s/h t ) ^ R-2c(T-S). 
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Proof. Let us assume by contradiction that U(s,s/h t ) < R\ — |c(T — S) for some R\ < R. We 
can assume also that if ht+i (or ht+i) is the deviation from ht (or ht), Le, the first coordinate of 
h t+1 (or /r +1 ) is different to s(hf) (or s(ht)) but the second one is equal to s(ht) (or s(ht)) follows 
that U(s,s/ht) — U(s,s/ht+{) (U(s,s/ht) — U(s,s/ht+i), respectively) is smaller than e with e 
chosen arbitrary small (provided 5 large and 1—p small). Now we take s* such that s*(ht) ^ s(ht), 
s*(ht) 7^ s(hf) and after that deviation s* is like s (observe that at this point we are using that s/ht is 
not symmetric, but we can do that because otherwise if for any t, s(ht) = s(ht) we can argue as in the 
proof of theorem [5]). Moreover, we also assume that U(s*,s*/ht) = R and U(s*,s*/ht) = R- From 
the assumption, follows c < B-I^b^hil anc [ so by lemma \T4\U(s. s/ht) <R\ — c{T — S). Therefore, 
U(s*, s/h t ) <Ri- 2c(T - S) and U(s*, s/h t ) < R 1 - c(T - S), so U(s, s*/h t ) < R x - c(T - S) and 
U(s,s*/hf) < R\ and therefore, 

U(s*,s*/ht) + U(s*,s*/h t ) - [U(s, s*/h t ) + U(s, s*/h t )} > R- R x + c(T - S) > R- R x , 

and since 

U(s, s/ht) + U(s, s/ht) - [U(s*, s/h) + U(s*,s/h t )} 

is arbitrarily small, by lemma [TT1 follows that s does not have a uniformly large basin of attraction. 

□ 

Note that in previous results only populations of two strategies were used. Now we explore 
consequences of considering groups of three strategies. 

Theorem 7. Given a strategy s, if there exists ht such that for any hk either containing ht or ht 
(either h t C hj. or h t C hk) follows 

U(s,s/h k ) < R x 

for some R± < R then s does not have a uniformly large basin of attraction. 

Proof. We are going to use theorem [2] which gives conditions, on group of three strategies, that 
implies that one of the strategies is an attractor but has an arbitrary small basin of attraction. 
More precisely, for s not to have a uniform large basin of attraction, it has to be shown that there 
exists Co > such that for any e > 0, there is s* and s' satisfying: 

(i) < U(s, s) - U{s*, s) = U(s, s*) - U(s*,s*) < e; 

(ii) < U(s, s) - U(s', s) = U(s, s') - U{s', s') < e; 

(iii) U(s*,s')-U(s',s')>C - 

(iv) U(s',s*)-U(s*,s*)>C . 

Observe that under that above conditions, it follows from theorem [2] and remark [6] that once we 
identify s with the vertex e\, the point ( (j +2e ' c +2e ) ^ s no ^ m ^ ne b asm of e\. 

Given ht such that for any hk that contains either ht or ht follows that U(s, s/hk) < R\ for some 
R\ < R, we can also take ht such that for the deviation h' t+1 from ht (i.e, the first coordinate of 
h t+1 is different to s(ht) but the second one is equal to s(ht)) follows that U(s, s/ht) — U(s, s/ht+i) 
is smaller than e with s chosen arbitrary small, provided 5 large and 1—p small. Moreover, the 
election can be done in such a way that the same holds for for ht- 

Now we build two strategies s*,s', that upset s. The strategy s* deviate respect to s at ht but 
coincide with s on hf. On the other hand, the strategy s' deviate respect to s at ht but coincide 
with s on ht- Both strategies coincide with s after the first deviation with s. In other words: 
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(i 
(ii 
(iii 

(iv 

(i 
(ii 

(iii 
(iv 

(v 
(vi 

(i 
(ii 

(iii 
(iv 

(i 
(ii 



s*(h t )^s(h t ) and S *(h t )=s(h t ); 
s'(h t ) = s(h t ) and s'(h t ) / s(h t ); 

h s*,s/h t (s*(h t ),sCht)) = h s,s/h t (s*{h t )Xht)) aIld h s*,s/ht(s*(ht),s(h t )) = h s,s/h t (s* (h t ),s{h t )Y 

h s',s/ht(s'(ht),s(ht)) = h s,s/ht(s'(ht),s(h t )) alld h s',s/ht{s'(ht),s(ht)) = h s,s/h t (s' (h t ),s(ht))~ 
Observe that from that properties follows: 

U(s,s*/h t (s(h t ),s*(h t ))) = U(s,s/h t ); 
U(s*,s*/h t (s*(h t ),s*(h t ))) = U(s,s/h t (s*(h t ),s(h t ))); 
U{s\s/h t ) = U(s,s/h t ), U(s,s*/h t ) = U(s\s*/h t ); 
U(s,s'/h t (s(h t ),s'(h t ))) = U(s,s/h t ); 
U(s',s'/h t (s'(h t ),s'(h t ))) = U(s,s/h t (s(h t ),s(h t ))); 
U(s',s/h t ) = U(s,s/ht), U(s,s'/ht) = U(s',s'/h t ). 
Therefore it follows that: 

U(s, s/h t ) - U(s*,s/h t ) < e and U(s, s/h t ) - U{s*, s/h t ) = 0, 
U(s, s/h t ) ~ U(s', s/h t ) < £ and U(s, s/h t ) - U(s', s/h t ) = 0, 

U(s*,s*/ht) - U(s,s*/ht) = -[U(s,s/h t ) - U (s*, s/ht)] and U(s*,s*/h t ) = U(s,s*/h t ); 
U(s',s'/ht) ~ U(s,s'/ht) = -[U(s,s/h t ) - U(s',s/h t )} and U(s,s/h t ) = U(s,s/h t ). 
Now we have to compare s' and s*. Observe that: 

K's* /h t (s>(h t ),s*(h t )) = h ss/h t (s(h t )Xh t )) so U{s',s*/h t ) is close to U(s*,s*/h t ), 
h ss>/h t (s*(h t ),s>(h t )) = h ss/h t (s(h t )Xh t )) so U ( s *i s 'l~ h t) is close to U(s',s/h t ). 



Since s* and s' deviate from s at ht and fit respectively and ht(s*(h t ), s'(h t )) is not one of the paths 
previously listed, we can assume that 

U(s*s'/ht(s*(ht),s'(h t )) = R. 

In the same way, 

U(s's*/ht(s*(h t ),s'(h t )) = R. 
Therefore, and from the assumption that U(s,s/hk) < Ri follows that 

U(s's*/h t ) + U(s's*/h t ) - [U(s*s*/h t ) + U(s*s*/h t )] > R — R\ — e 

and 

U{s*s'/h t ) + U{s*s'/h t ) - [U{s's'/h t ) + U{s's'/h t )] >R-R±-e 
and therefore the theorem is proved. 

□ 
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From the proof of previous theorem it can be concluded the following (details are left for the 
reader) : 

Theorem 8. If s has a uniformly large basin of attraction and let ht such that U(s,s/ht) is suffi- 
ciently close to inf{U(s,s/hk) '■ V/t&}. Then it follows that either 

U(8,8/ht(a(ht),l(hd) > R 

or 

U(s,s/h t (s(h t ),7(htj) > R, 
where s{h^) = D if and only if s(hk) = C. 



7 Revisiting the sufficient conditions to have a uniformly large 
basin 

In this section we provide general sufficient conditions to guarantee that a strategy has a uniformly 
large basin (see definition [3]) , i.e., conditions that implies that a strategy has a uniform large basin 
of attraction independent of the initial population, for large discount factor and small trembles. 
This is based in theorem [TJ In subsection 17.11 we introduce another type of condition which is easier 
to calculate than the previous one and also implies that a given strategy satisfying it has a uniform 
large basin of attraction. 

Given two strategies s\ and s 2 , to avoid notation, we write 

Ns, p (si,s 2 ) := Us yP (si, si) - Us, p (s 2 , s{). 

Let s be a subgame perfect strategy. Given s' and s* with Ns^ p (s,s*) ^ Ns p (s,s') we consider the 
following number 

,, , * N 5 ^(s,s*) + N s , p {s,s') + Us, p {s',s*)-Us, p (s,s*) + U 5 ^s*,s')-Us, p {s,s'), 

M StP {s, s , s ) := — £ — — f- . 

N Sp (s,s*) 



M s<p (s):= sup {Ms, p (s,s*,s'), 0}. 

Ns, p (s,s*)ZN s , p (s,s>) 

Remark 8. If we take the payoff matrix associated to a set of strategies that includes s,s*,s' and 
s = ei, s* = ej, s' = Cj it follows that Ms >p (s, s* , s') = Ma+^g as { n lemma[J\ and theorem] 101 

Remark 9. Observe that in the case that s* = s' , the quantity M§ p (s,s*,s') is equal to 

2[N s>p (s,s*)+N s>p (s,s*)} 



2M SiP (s,s*). 



So, for the purpose of bounding Mg p (s) from +oo it is enough to take the supreme over M§ p (s, s* , s'). 
Observe also that if we only consider the population {s,s*} then the segment [0, j^jjiprj) is in the 
basin of attraction of s (provided that s is identified with e\). 

Definition 8. We say that a strategy s satisfies the "Large Basin strategy condition" if it is a 
subgame perfect strategy and if there exist 5q and Mq such that for any 5 > So and p > p{5) there 
exists Mq{6) verifying 

M S)P {s) < M (6) < oo. 
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We can also define 



M(s) := limsupM 6jP ( 5) (s) 
5,p— >1 



and observe that in this case, if M{s) < oo then s has a large basin of attraction (but the size could 
depend on S and p). 

Remark 10. It is important to remark that it could hold that limsup^^ sup s * {Mg p rg\(s, s*)} < +00 
but M(s) = +00. This means that to guarantee a uniform Li—size basin in any population, it is not 
enough that a strategy has uniform size of basin against any other strategy. 

Definition 9. We say that a strategy s satisfies the " uniformly Large Basin condition" if it is a 
strict subgame perfect strategy and 

M(a) < 00. 

Theorem 9. If s satisfies the "uniformly Large Basin condition", then s has a uniformly large basin. 
More precisely, let (3 be small. Then, there exists 5q such that for any 5 > 60 (p > p(5) ) and any 
finite set of strategies S containing s, follows that s is an attracting point such that 

B(s) C Bf oc (s) 

where 

B(s) = {Oi, ...,x n ): x 2 + ■ ■ ■ + x n < + ^ } 

and n = cardinal{S). 

Proof. The proof follows immediately from theorem Q] and the definition of M{s). In fact, ordering 
the strategies in such a way that s corresponds to the first one and N(s, S{) N(s, Sj) if j > i then 
it follows that for 5 large, then the constant Mq = sup{ M 'j^ M i* ? q} < M{s) + (3 and therefore B(s) 
is contained in the basin of attraction of e\. □ 

Remark 11. Observe that to guarantee a uniform size of the basin of the attraction independent of 
the population, it is enough to bound a condition that only involves pair of strategies. 

Remark 12. Given a strict subgame perfect strategy s and a population S, the lower bound of the 
size of the basin of attraction of s can be improved by taking 

Ms, p (s,S):= sup {M s ,p(s,s*,s'), 0}. 

N S:P (s,s*)^N StP (s,s'),s',s*eS 

To check that Mg jP (s) < +00 observe that 

M 5jP (s,s*,s') 

N SjP (s, s*) + N s>p (s, s') + U StP (s', s*) - U 6)P {s, s*) + U s , p (s* , s>) - U s , p {s, s') 

N 5 , p (s,s*) 

i N s , p (s,s') U s , p (s', a*) - U s , p (s, s*) + U s , p (s*, a') - U s , p (s, a') 



Then if 



defining 



N S , P (s,s*) N s , p (s,s*) 
7 1 * /x _ Us, p (s',s*) - U s , p (s,s*) + U s , p {s*, s') - U s>p (s, s') 



Z s , p (s) := sup {Z s , p (s,s*,s')} 

N s ,p(s,s*)>N S:P (s,s>) 
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and using that ^' p ^'^ ^ 1 then follows that M$ )P (s) < +00 if and only if Z$^ p (s) < +00. 

In other words, s is a "Large Basin strategy" if and only if Zg p (s) < +00. Similarly, defining 

Z{s) := limsup Zs, p (s), 
5,p(<5)->l 

s is a "uniform Large Basin strategy" if and only if 

Z(s) < +00. 

Question 1. Is the uniformly large basin condition (recall definition^ a necessary condition for a 
strategy to have a uniformly large basin strategy? 

7.1 Asymptotic bounded condition 

We provide now a condition that implies that s is has a uniformly Large Basin of attraction. This 
new conditions are based on the conditions defined before but are easier to calculate. Moreover, if 
a strategy satisfies them it follows that has a uniformly large basin of attraction. 

Definition 10. We say that a strict subgame perfect strategy s satisfies the asymptotic bounded 
condition if 

- there exists R® such that for any s* holds 

r Us, p (s,s)-U s (s,s*) 

limsup sup — — < Rq, (32) 

8->l,p-H,p>p(8) s*:Ng !P (s,s*)>0 ISI S,p{S,S ) 

- there exists R\ such that for any s*,s' for which Ns :P (s,s*) ^ N$ tP (s,s*) holds, then 

v Us,p(s',s*) + UsAs*,s')-2U s , p (s,s) 

limsup sup — mi *\ < (33) 

8-*l&-*\,p>p(5) s*:N s (s,s*)>0 Ns :P (S, 8*) 

Theorem 10. Let s be a strict subgame perfect strategy satisfying the asymptotic bounded condition. 
Then, s has a uniformly large basin of attraction. 

Proof. Recalling that Ns^ p (s, s') ^ Ns tP (s, s*) we need to bound by above the following expression 

U 5 ,p(s', s*) - Us, p {s, s*) + U 5 ,p{s*,s r ) - U 5 ,p(s, s') 



So, 



Us,p( s> i s *) ~ t^pjs, s*) + Us, P (s*, s') - Us, p (s, s') 

N s , p (s,s*) 
Us, P (s', s*) + U S)P {s*, s') - 2U s , p (s, s) 



Ns, P (s,s*) 



+ 



Us,p( s > s ) ~ U S,p( s ' s *) , U 8,p( s ' s ) - U 8,p( s ' s ') < 

N s ,p(s,s*) N SiP (s,s*) 
<R Us, P {s, s) - Us, p (s, s*) Us, P (s, s) - Us, P {s, s') N StP (s, s') 
" 1 N 5:P (s,s*) N s ,p(s,s') N SjP (s,s*) 

R\ + 2Rq. 



□ 
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From now on, we denote 

N 5 , p (s, s*) := U 5 , p (s, s) - U s , p (s, s*) (34) 

B 6ip (s, s*,s') := U 5 , p (s', s*) + U s , p (s*, s') - 2U s , p (s, s) (35) 
Remark 13. From the proof of theorem \7U follows that M(s) ^ 2 + 2Rq + R\. 

7.2 Having uniform large basin for population of two strategies is not enough 

In this section we give an example that shows that when a population of three strategies are 
considered it can happen that one of them has a uniformly large basin when we consider the 
subset of two strategies but it does not have a large basin when the three strategies are considered 
simultaneously. In other words, next theorem shows that the example given in theorem [2] can 
be obtained as the replicator equation associated to three strategies. In what follows, given a 
population of three strategies S = {s, s*, s'} and its replicator equation (in affine coordinates), the 
first strategy is identified with the point (0,0). In the theorem below we considered the repeated 
prisoners' dilemma without tremble and the proof in trivially adapted for the case of trembles 
provided small probability of mistakes. 

Theorem 11. For any A small, there exists a population of three strategies S = {s, s*, s'} such that 
(i) s is an attractor in S; 
(ii) s always cooperate with itself; 

(Hi) in the population {s,s*}, s is a global attractor (in the terminology of the replicator equation, 
the interior of the simplex associated to {s,s*} is in the basin of attraction of s); 

(iv) in the population {s,s'} s is a global attractor; 

(v) the region bounded by H\, V\ and X2 + x% = 1 does not intersect the basin of attraction of s. 

Proof Given any small A > 0, we build three strategies such that identifying s with (0,0), s* with 
(1, 0) and s' with (0, 1) satisfy the hypothesis of theorem[2j We also assume that the strategies s' and 
s* deviate from s at the 0— history, s plays always cooperate with itself and so s'(0) = s*(0) = D. 
We fix 7 > and we take e small. Observe that provided any e > small, taking 5 large, follows 
that there exist different b^, b' 2 ,b' 3 , b' A and b\, 6|, 63, b\ such that 

< R - (b[R + b' 2 T + b' 3 S + 64P) = R - {b\R + b\T + b\S + b%P) = e 

but 

R - (b[R + b' 2 S + b' 3 T + b' 4 P) = R- (b\R + b* 2 S + b* 3 T + b* 4 P) > 7. 
Now, from (C, D) we choose s, s', s* such that 

U s (s, s*) = U 5 (s, s') = b' x R + b' 2 T + b 3 S + b' A P 

but in such a way that s' 7^ s*. To show that it is possible to choose s' independently of s* against 
s is enough to take s'(C,D) 7^ s*(C,D). Now, we take s* and s' from [D,D) such that 

s*(D,D)^s'(D,D) 
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and 

U s (s*, s*) - U s (s, s*) = U 5 (s*, s*) - (b$R + b* 2 S + b$T + b\P) = -e, 
U 5 (s', a') - U 5 (s, s') = U 5 (s', s') - (b[R + b' 2 S + b' 3 T + b' A P) = -e. 
Moreover, we can take s', s* such that 

U s (s',s*) = U s (s',s*)=R 

therefore, 

U s {s', s*) - U s (s*, s*) = U s (s>, s*) - U s (s', s') > 7. 

Us(s',s*)-Us(s*,s*) > 7 
Us(s,s) -Us(s*,s) e 

and so choosing e properly we can assume that the quotient is equal to j. □ 

Remark 14. Previous results can also be proved using a generalized versions of the Folklore's 
theorem: Any payoff matrix can be realized as the payoff matrix of a finite set of strategies, provided 
that 5 is large, the entries of the matrices are in the admissible range of payoff and the entries aij,aji 
satisfy the relation given by lemma® 



8 Recalculating payoff with trembles 

Now, we develop a way to calculate the payoff for certain strategies which roughly speaking consists 
in approximating the payoff using equilibrium paths, provided that the probability of mistake is 
small. This first order approximation allows to prove the asymptotic bounded condition (see in- 
equalities (f32|) . (|33p . (|38p . (|40p and lemma [TTj) for certain types of strategies (namely strict subgame 
perfect strategies, see definition [TT1) . In few words, the difference in utility between two strategies 
can be estimated in the following way (provided that p is sufficiently close to 1): 

• first, we consider all the paths (on and off equilibrium) up to its first node of divergence 
between the two strategies, namely h^jhk (see equalities (|36l 1371 [39]) ) . 

• from the node of divergence we only consider equilibrium payoffs (see lemma [16] ) . 

In particular, if s(ho) 7^ s*(/io) then Us :P (s,s) — Us :P (s*,s) is approximated by Us tPt h s , s (s, s) — 
U StPths „ a (s*,s). 

More precisely, recalling that 

Ns, p (s,s*)= ^2 5 k VsA h k)\U8, p {s,s/h k h k ) -U s , p (s*,s/h k h k )]. (36) 

h k ,h&1Z* ss , 

N s ,p(s,s*)= 8 k p SiS (h k )[Us tP (s,s/h k h k ) - Us, p (s,s* /h k h k )}. (37) 



h k ,hen* st 



we define 



*$p(*.0 := E 5k PsAhk)[U 5 , p (h s s/hkhk ) - U s>p (h^ s/hhhk ) 



h k ,hen* . 
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Nl P (s,s*) := Yl 5k Ps,sihk)[U S , p {h ss/hkhk ) - U 5 , p (K tS , /hkhk )} 

h k ,heTL* ss , 

where given strategies S\,S2 

u sA h s US2 /h k h k ) := u s,p( h si,s 2 /h k ) + U s , p (h SljS2/hk ). 
We look for conditions such that there exists a uniform constant C satisfying that 

We develop a similar approach Bs p (s, s', s*) (see equation (|35p ) that consists in comparing different 
paths for three strategies s,s*,s'. Given any pair of paths h,h where s,s',s* differ (meaning that 
at least two of the strategies differ at some finite paths contained either in h or h), there exist k' = 
k(s, s' , h),k' = k(s,s',), k* = k(s, s* ,h),k* = k(s,s*,h), such that s(h k /) ^ s' (h k r), s(h k >) 7^ s'{hy) 
and s(h k *) 7^ s*(h k *). Observe that some of them could be infinity. 
We take 

k(s,s',s*) := mm{k',k', k*,k*} 

which is finite and observe that 

Pss(hk) =p s 's*(h k ) = p s » s ,(h k ) = Ps , s (h k ) =p s 's(hk) 
Pss(hk) =p s 's*(h k ) =p s * s '(h k ) =p s * s (h k ) =p s ' s (h k ). 

so 

B S}P (s, s*, s') = ^ S k p ss (h k )[U StP (s', s*/h k h k ) + U s , p {s* , s' /h k h k ) - 2U SiP (s, s/h k h k )). (39) 

h:k(s,s\s*) 

Now we define 

Bl p (s,s*,s') = Yl 5k P^h k ){U s , P {h s , s , /hkh ) + U s , p (h s ^ sl/hkkk ) - 2U S)P (h ss/hkhk )}. 

h:k(s,s' ,s*) 

So, in a similar way we look for conditions such that there exists a uniform constant C 

B s>p (s,s*,s') Bl p (s,s*,s>) 
N SjP (s,s*) " Nl p (s,s*) ■ { ' 

We are going to restrict a relation between p and 5. From now on we assume that 

p ^ y/5. (41) 

Moreover, and to simplify calculations we change the usual renormalization factor 1 — 5 by l ~ p2 - 
and so we calculate the payoff as following: 

1 ^ S 

Us, p (s 1 ,s 2 ) = 2 — ^2 5 t Ps 1 ,s 2 {at-,h)u{a\b t ). 

Both ways calculating the payoff (either with renormalization 1 — 5 or S ) are equivalent as they 
rank histories in the same way. In addition it holds that: 

1 1 ~ 6 

2 < 1 - bp 2 < 
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Observe that if s± = s 2 along the equilibrium it follows that 



Lemma 15. It follows that 



N S , p (s, 3*) ^ Nl p (s, s*) + 2 I P M AI: 



p 2 {l-5) 
l-p 2 



p 2 {l-5) 



M: 



B StP (s, s*, s') ^ Bl p (s, s*, s') + 3 ^ ^ M. 

The next definition is an extension of the definition of subgame perfect strategies. 

Definition 11. We say that s is a uniformly strict sub game perfect if for any s* follows that given 
h G 1Z StS * then 

(1 - p 2 5)C < U s , p (h SjS/hh ) - U s , p {h s * tS/hk ), (42) 

for p > po, 5 > 5o where Co, 5q,pq are positive constants that only depend on T, R, P, S. 

Given 5 we take p such that it is verified, 

l-p 2 M 
p 2 (l - 5) C (l -p 2 5) 

Since p < 1 follows that 1 — p 2 5 < 1 — 5 and taking p > \ then to satisfies (j43l) we require that 

3 l-p 2 M 



4 (1 - 5) 2 Co 

Therefore, we take 



< 1. (44) 



and observe that it is a function smaller than 1 for 5 < 1. Then, we define 



4 0),, n2 



p(S) = max{-,pi(S),V6} (45) 



Lemma 16. //s* is strict subgame perfect and p > p(5) (giving by equality \4^\) th 

N s , p (s,s*) ^ N$ p (s,s*) + ' 
B s , p (s,s*,s') < Bl p (s,s*,s') | i 



(7/ 
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Proof. It follows from lemma [T5l s is a subgame perfect and that inequality (|43p is satisfied 



N s , p (s,s*) 



^( S)5 *)(1 + 2M (1 _^ 1 _^ ) ) (1-^0,(1-^ 



+ 1. 



In a similar way it is done the estimate for ~jf~~^~^y- d 
Now we will try to estimate Y r 6 ' p \ s ' s \_Y r 6 ' p \ s * I based on lemma [TBI 



Lemma 17. If p > p(S) (giving by equality \Jd^ and s is a uniform strict and there exists D such 
that for any h E 1Z* s * holds 

u sAK,s/h k k k ) - Us, P (K,8*/h k h k ) < g 

U sA h s,s/h k h k ) ~ U sAK*,s/h k hk) 

then 

Us,p( s > s ) ~ u sA s > s *) <D + 1 
Us )P (s,s)-U s>p (s*,s) 

Proof. It is enough to estimate ^' P ^' S J 
NsA 8 , 8 *) 

Ehen s , s «s,pS k PsAhk)(U5A h s, s /h k h k ) ~ U *Ah S)S * /hk ~ hk )) 
Ehen s>s *,5,P 6kps A h k){UsA h s, s /h k h k ) ~ u sA h s*,s/h k h k )) 

rfc / 7 S Us >p( h s,s/h k h k }~ U 6,p( h s,s* /h k h k ^ fjj /, \ _TT (h \ 

_ ^ k > h k° P»A n k) U SiP (s,8/h k h k )-U s , p (s*,s/h k h k ) ^ Us ^ n s,s/h k h k ) U &A n s, s */h k h k > 

Ek,h k 6k PsAhk)(u 5 , p (h s s/hkhk ) - UsAh s ^ s/hk h k ) 

5,p 5k PsA h kWsA h s , s /h k h k )- U sA h s*,s/h k h k )) _ „ 

Ehen s , s *,s >P sk PsA h k)(UsAK, s /h k h k ) - u sAK*,s/h k h k )) 



□ 



9 Existence of strategies with a uniformly large basin of attraction 

In the present section we show that strategies like win-stay-lose-shift satisfy the conditions intro- 
duced in subsection 17.11 

Definition 12. win-stay-lose-shift Let us define the strategy known as win-stay-lose-shift: f it 
gets either T or R stays, if not, shifts. From now on, we denote win-stay lose-shift as w. 
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For win-stay-lose-shift to be a subgame perfect strategy it is required that 2R > T + P (see 
|NSJ and [RCJ). The next lemma is obvious but we state it since is fundamental to do a series of 
calculations related to w. 

Lemma 18. Given a finite path ht it follows that w is a symmetric strategy, meaning that 

w(h t ) = w(h t ). 

Proof. It follows from the fact that 

w(C,D) = w(D,C) = D. 

□ 

Theorem 12. If 2R > T + P then w has a uniformly large basin. 

First we prove that w is a uniform strict subgame perfect (this is done in subsection 19.0. ip . and 
later we show that w satisfies the "Asymptotic bounded condition". For the latter we need to bound 



(46) 



(47) 



and 

Bs tP (s,s*,s') 

this is done in subsection 19.0.21 and 19.0.31 respectively. 

9.0.1 w is a uniformly strict subgame perfect. 

Given we have to estimate 

) - Us >p (h SiW /h k ) 

where h WjW /h k is the equilibrium path for w, w starting with hk and h B}W /fi k is the equilibrium path 
for s,w starting with hk- 

In what follows, to avoid notation, with U(., .) we denote f7$ iP (/i ,/h fc ). Following that, we take 

E fc-i^ E 

j:vj(s,w/h k )=R ' j:vj(s,vi/hk)=S 

j:ui(s,w/h k )=T ' j:ui (s,w /h k )=P 

Observe that 

b 1 + b 2 + b 3 + b 4 = l 

and 

C/(s, u>) = hR + 6 2 <S + + b A P. 

From the property of w, for each T that s can get (s plays D and w plays C) follows that in the 
next move s may get either S or P because w plays D, so, 

&2 + &4^p 2 ^3- (48) 
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To calculate U(w,w) we have to consider either s(hf~) = C, w(hk) = D or s(hk) = D, w(hf~) = C. 
So, from lemma [151 

r r a w(h k ) = c 

U ^ = { igip+jtSR ifw(h k ) = D 

To calculate U(s,w) in case that s(h k ) = D,w{hk) = C, writing R = b±R + b 2 R + b^R + b 4 R by 
inequality ([48 P it follows that 



U(w,w)-U(s,w) = b 2 {R - S) + b 3 (R - T) + b 4 (R - P) 

> (b 2 + b A ){R-P) + b 3 {R-T) 

> 8p 2 b 3 (R-P) + b 3 (R-T) 

> b 3 [(l+p 2 5)R-(T + P)}. 

Observing that if s(hk) = D,w{h k ) = C, then 

b 3 > 1 - P 2 5 

and since 2R — (T + P) > it follows that for 5 and p large (meaning that they are close to one) , 
then [(1 + p 2 5)R — (T + P)] > Co for a positive constant smaller than 2R — (T + P) and therefore 
(provided that 5 and p large are large) follows that 

U{w,w) - U{s,w) > (l-p 2 5)C , 

concluding that w is a uniform strict subgame perfect in this case. 

In the case that s{hk) = C,w(hf.) = D, observe that b 2 ^ 1 — 5 and calculating again the 
quantities bi,b 2 ,b 3 , b 4 but starting from j ^ 1 then we get that 

U(s,w) = (l-p 2 5)S+p 2 5[b 1 R + b 2 S + b 3 T + b 4 P]. 

Therefore, writing p 2 5R = p 2 5\b\R + b 2 R + b 3 R + b 4 R] and arguing as before, 

U(w, w) — U(s, w) 





(1 


-P 2 5)(P 


-S)+5[b 2 {R-S) + b 3 (R-T) + 


b 4 (R-P)} 




(1 


-P 2 S)(P 


-S)+5[(b 2 + b 4 )(R-P) + b 3 (R 


-T)) 




(1 


-p 2 5)(P 


-S) + 6[6b 3 (R-P) + b 3 (R-T)\ 






(1 


-p 2 6)(P 


-S) + 6b 3 [(l + S)R-(T + P)} 





since 2R — (T + P) > it follows that for 5 large (63 now can be zero) 

U(w, w) - U(s, w)>{1- p 2 5)(P - S), 

proving that w is a uniform strict subgame perfect in this case. 

Remark 15. Given e small follows that for 5 large then Co can be estimated as 

C = min{P- S,2R- (T + S) -e}. (49) 

Remark 16. To prove that w is a uniform strict subgame perfect, the main two properties of w 
used are 

(i) it cooperates after seeing cooperation and so U(w,w) = R after w(hk) = C, 
(ii) after getting P it goes back to cooperate, so U(w,w) = (1 — 5p 2 )P + 5p 2 R after w(hk) = D, 
(Hi) it punishes after getting S, 
(iv) 2R>T + P. 

Observe, that the previous calculation does not use that w keeps defecting after obtaining T. 
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9.0.2 Bounding fl46|) . 

First we estimate Us tP (w,w) — Ug iP (s,w) and Us :P (w,w) — Us tP (w,s). Recall that from lemma [T71 it 
follows that is enough to bound for any h 6 TZ^ s : 

Calculating numerator and denominator. 

For the moment, to avoid notation, we denote 

U(s,s') := Us, p (h ssl/hkhk ) = U s , p (h StS , /hk ) + U{h s sl/hk ). 

Observe that if 17(10, w) - U(s, w) = B 2 (R- S) + B 3 (R -T) + B A (R- P), then 

U(w,w) -U(w,s) = B 2 (R-T) + B 3 (R-S) + B 4 (R-P). 

To avoid notation, let us denote L = U(w, w) — U(s, w) = B 2 {R — S) + B 3 (R — T) + B^(R — P) so, 
B 4 (R -P) = L-[B 2 (R- S) + B 3 (R - T)] and therefore 

U(w,w) -U(w,s) = B 2 (R-T) + B 3 (R- S) + L-[B 2 (R- S) + B 3 (R-T)] 
= L + B 2 (S - T) + B 3 (T - S) 
= L + {B 3 -B 2 )(T-S) 
«C L + B 3 (T-S) 

recalling that in case that b 3 / then L = U{w, w) - U(s, w) ^ B 3 [(l + 5)R - (T + P)] (if B 3 = 
then utZ]-UM < !) ^ follows that 

U(w,w) -U(w,s) L + B 3 (T-S) 

U(w,w) — U(s,w) ^ L 

B 3 (T-S) 



< 1 + 
= 1 + 



B 3 [(1 + 5)R-(T + P)] 
T-S 



Therefore, 



(1 + S)R - (T + P) 

U S,p{^w,w/h k 'h k ) ~ Us A h w,s/h k h k ) T-S 



^1 + 7- p-= j— =r, (50) 



so by lemma [T71 

Us, P {w,w) - Us, p (w,s) < T - S 

Us^(w,w)-Ue^8,w) " (1 + <5)fl - (T + P) 

Remark 17. T/ie main property of w used to bound j^6[ ) is £/iaf ifb 3 ^0 th 

U(w, w) - U(s, w) > 6 3 [(1 + - (T + P)} 

and this follows from the properties listed in remark [TR 



en 
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9.0.3 Bounding ([37]) 

By lemma [TBI we need to bound 

Bi p (s,s*,s>) 

Recall that 

Bl p (s,s*,s') = Yl 6k Pss(h k ){U 5 , p {h s ,^ /hkhk ) + UsAK* tSl/hkkk ) ~ 2U sAK,s/h k h k ^ 

h:k(s,s',s*) 

For the particular case of s = w we divide the paths in two types: either = C or w(hk) = D. 

In the first case we claim that 

U sA h s ',s*/h k h k ) + U S,p(K*,s'/h k h k ) ~ 2U sA h w , W /h k h k ) < °- 
Observe that U$ )P (h w w / hk f lk ) = 2i? and by lemma [7] follows the assertion above. Therefore, 

Bl p (s, s*,s') < Yl U Uh>, s */h k h k ) + U S, P (K*,s>/h k k k ) ~ 2U sAh w , w/hkhk ). 

h:k(s,s' ,s*),w(h k )=D 

In case that w{hk) = D observe that U{h ww , h j lh ) = 2 1 ~^ - P + 2R5. To deal with this situation 
we consider two cases: i) s'(/ifc) = C or = C. and u) s*(/ifc) = (7 or s*(/ifc) = C. So, 

h:s'{h k )=C Or s'{h k )=C 

E U S,p(K',s*/h k h k ) + U 5A h s*,s>/h k h k ) ~ 2U $A h w,w/h k h k y 

h:s*(h k )=C Or s*(h k )=C 

Case i) s'(hk) = C or s'(hk) = C: In this situation follows that /i £ 1Z*(s', w). We rewrite 
E U sA h s>, s */h k h k ) + U s,pih s *,s'/h k h k ) - 2U sA h w , w /h k h k ) = 

h:s'(h k )=COTs'(h k )=C 

E U S,p( h s',s*/h k ) + U &A h s*,s'/h k ) ~ U 5,piK,w/h k h k ) + 

h:s t (h k )=COTs'(h k )=C 

E U S,pi h s*,s'/h k ) + U S,p{ h s',s*/h k ) ~ U S,p(. h w,w/h k h k )- 

h:s'(h k )=COTs'(h k )=C 

Using that /i G 1Z*(s',w), and again lemma [TJthen 



E u s A h s', s */h k ) + u s,p{ h s*, S '/h k ) ~ u sAK,w/h k h k ) < 

/i:5'(h fc )=Cors'(ft fc )=C? 

'" /r/i E *w(W[S + r-2P] 



and 



2 

^ h:heK*(s',w) 



E U 8,p( h s*,s'/h k ) + U sA h s>,s*/h k ) ~ U sA h w,w/h k h k ) < 

h:s'(h k )=C OI s'(h k )=C 

^ h:h£K*{s',w) 
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but since 

Us, p (w,w)-Us, p (s',w)^^^ P w Ah k )5 k [2P - {S + P)] 

P h-.h&K* (s' ,w) 

follows that 

Y U SiP (h s , tS * /hk ) + Us !P (h s * jS , /hh ) - U 5>p (h w>w/hkhk ) ^ U s , p {w,w) - U StP {s',w) 

h:s'(h k )=C OT s'(h k )=C 

Yl U s , p (h s » tS , /hk ) + U 5 , p (h s , tS * /hk ) - U s , p (h w w/hkhk ) < Us, p (w,w) - U s , p (s',w). 

h:s'(h k )=C Or s'(h k )=C 

Case ii) s*(hk) = C or s*(hf.) = C: In this situation follows that h £ lZ*(s*,w), and using this 
key statement we conclude in a similar way that 

Y U 5>p (h s * >sl/hk ) + U s>p (h s ^ s , /hk ) - U SjP (h w w/hkhk ) < 

h:s*(h k )=C Or s*(h k )=C 

< U 5jP (w,w) - U s>p (s*,w) 

Y u s , p (h s * iS , /hk ) + u SiP (h s , ^ /hk ) - u s , p (h w w/hkfik ) ^ 

h:s* (h k )=C Or s* (h k )=C 

< Us, p (w,w) - U 5>p (s*,w). 
Therefore, recalling that 

Us, p (w,w) - Ug tP (s*,w) ^ Us, p {w,w) - Us jP (s',w) 

we conclude that 

Bs, p (s,s*,s') 
Us, p (w,w) - U s , p (s*,w) 

is uniformly bounded and therefore bounding (|47p . 
9.1 Generalized w for any payoff system 

Recall that w has uniformly large basin, provided that 2R > S + T. Now, we consider w— type 
strategies that have a uniformly large basin for any payoff system. 

Definition 13. n-win-stay-lose-shift n— win-stay lose-shift. If it gets either T or R stays; if it 
gets S, shifts to D and stays for n— period and then acts as w. We denote it with w n . 

Theorem 13. For any payoff set there exists n such that w n is has a uniformly large basin. 

Proof. The proof follows the same steps that we used to prove that w has a uniformly large basin 
of attraction when 2R — (T + P) > but using the fact that for any payoff matrix there exists n 
such that 

nR > T + (n - l)P. 

To show that w n has a uniformly large basin of attraction, we calculate the quantities b\, 62 > 03,04 
for u(s, w n ) as it was done for w in subsection 19. 0.1 1 In addition, observe that for w n it follows that 

b 2 + b^5p" 1 -^f ) 2 n b 3 
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and if n is large enough then fzj~z > n — 1 and therefore, 

b 2 + b A > (n- 1)6 3 . 

Repeating the same calculation done for to, in case w n {h^) = C, s{hk) = D follows that 

U{w n ,w n ) - U{s,w n ) > {n - l)b 3 {R - P) + b 3 (R - T) > (1 — 5p 2 )[nR — T — (n — 1)P]. 

In case w n (hk) = D, s{h k ) = C, the calculation is similar. 

To bound uniformly the quantities (j46j) and (|47j ) for w n , we proceed in a same way that was 
done for w and it is only changed the upper bound 2R — (T + S) by nR — T — (n — 1)P. 

□ 



9.2 Examples of strategies with low frequency of cooperation which have large 
basin but they do not have uniformly large basin 

In what follows, we give examples of strategies with arbitrary low frequency of cooperation which 
have large basins (with size depending on S and p), however, those strategies do not have uniformly 
large basin of attraction. In other words, the lower bounds of their basin shrinks to zero when 
S,p — > 0. More precisely, they can not have uniformly large basin due to theorem [H Those 
strategies are built combining w with a. Moreover, we establish some relation between the frequency 
of cooperation and the lower bounds of the size of their local basin (but depending on 5 and p). 

Definition 14. We take n large and bo < 1, we define the strategy aw n,b ° as the strategy that in 
blocks of times I l w = [l(n + mon),l(n + mo«) + n — 1] behaves as w and in the blocks of times 
l\ = [l(n + uiqu) + n, (I + l)(n + iriQn) — 1] behaves as a, where mo denotes the integer part of 5- 
and I is a non-negative integer. 

Theorem 14. For any n large, and any positive bo the strategy aw n ' b ° has a large basin of attraction, 
but not a uniformly large basin of attraction. 

Proof. From now on, and to avoid notation, we denote aw n ' d ° with aw. First we are going to prove 
that aw is a strict sub game perfect. 

The strategy aw is a uniform strict subgame perfect: The proof is similar to the one performed 
for w. Let s be another strategy and given a path h let k be the first deviation (s(hh) 7^ aw(hk)). 
Either k £ r w or k € P a ] for some non- negative /. It follows that 



Us,p{h aw ,aw/h k ) = b R + (1 - b )P 



where 



j^0:ui(aw,aw/h k )=R j>0,I l w 

Observe that provided 5 large, then bo is close to do- Now we take s and assuming that it differs 
in hk and aw{hk) = R,s{ht) = D. In what follows, to avoid notation, with U(.,.) we denote 
Us,p,h (•) -/hk)- Following that, we take 

j:ui (s,aw/h k )=R j:ui (s,aw/h k )=S 
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j:ui (s,aw/h k )=T j:ui (s,aw /h k )=P 

Observe that 

h\ + b 2 + 63 + b 4 = 1 

and 

U(s, w) = biR + b 2 S + b 3 T + b 4 P. 
Moreover, since in blocks I l a aw behaves as a then 

64 > 1 - 6 (52) 

From the property that aw behaves as w in blocks of the form [l(n + m^n), (I + l)(n + rriQn) + n], 
for each T that s can get on those blocks (s plays D and it) plays C) follows that in the next move 
s may get either S or P because w plays D, so, noting 

= ^ E p" +2 * 

jei l w .u3{s,w/h k )=p 

then 

64 ^ 1 - 60 + C (53) 
b 2 + b%>p 2 5b 3 . (54) 

Writing 

1/(010, aw) = b R + (1 - 6o)-P = [bo - (1 - &4)]-R + + b 2 R + 03-R + (1 - b )R 
by inequalities ([52l [53l |54|) it follows that 

[/(aw, aw) - £f(a, aw) = [b - (1 - b 4 )]R + b 2 (R - S) + b 3 (R - T) + (1 - b - b 4 )(R - P) 

> (b + b 4 -l + b 2 )(R- P) + b 3 (R-T) 

> (6? + b 2 )(R-P) + b 3 (R-T) 

> 5p 2 b 3 (R-P) + b 3 (R-T) 

> b 3 [(l+p 2 5)R-(T + P)}. 

Observing that if s(hk) = D, aw(hk) = C, then 

63 > 1 - P 2 6 

and since 2R — (T + P) > it follows that for 5 and p large (meaning that they are close to one) , 
then [(1 + p 2 5)R — (T + P)] > Co for a positive constant smaller than 2R — (T + P) and therefore 
(provided that 5 and p large are large) follows that 

U(aw, aw) — U (s, aw) > (1 — p 2 S)Co, 

concluding that w is a uniform strict subgame perfect in the case aw{hk) = C, s(hf~) = D. 
In the case that s(hk) = C, aw(hk) = D so we know that 

U(aw, aw) = 1 ~f 5 P + p 2 5[b R + (1 - b )P] 
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where 6q is calculated as in (|5ip . but starting from j = 1. Calculating again the quantities b\, b^, 64 
but starting from j ^ 1 then we get that 

U(s, aw) = (1 - p 2 5)S + p 2 5[b 1 R + b 2 S + 63^ + b 4 P}. 

Therefore, writing 

p 2 5[b R + (1 - b )P) = p 2 5[b - (1 - 6 4 )]-R + hR + 6 2 i? + 63^ + (1 - b )R] 
and observing that also holds inequalities (|521 [53l |54|) and arguing as before it follows that 

U(aw,aw) - U{s,aw) ^ (1 - p 2 5)(P - S) + 5b 3 [(l + 5)R - (T + P)} 
since 2R — (T + P) > it follows that for 5 large (63 now can be zero) 

U{aw, aw) - U{s, aw) > (1 - p 2 5)(P - S), 

proving that aw is a uniform strict subgame perfect in the case aw(hk) = D, s(hk) = C. 

The strategy aw verifies the asymptotic bounded condition, but depending on 5p 2 : Bounding J^6p 
and for aw: To bound Us !P (s, s) — Us tP (s, aw) we repeat the argument done for w and observe 
that the key point is that U (aw , aw / h^) — U(s, aw/h^) > b^((l + 5)R — (T + S)) which has been 
proved when is proved that aw is a uniform strict subgame perfect. 

To bound (|47p we perform the same approach for w, however the estimates changes depending 
on do. More precisely, given s' and s* follows that 

B(s', s*,aw) ^ 2(1 - do)(R - P), 

therefore, arguing as in the case of w follows that 

B SjP (s',s*,aw) 2(1 -d )(R-P) 
N 5 , p (aw,s*) " (l-p 2 5)(P-S) - 

The strategy aw does not a uniformly large basin of attraction: It follows from the fact that aw 
is symmetric but no efficient. 

□ 



10 Perturbed Replicator Dynamics 

We consider more general equations than the replicator dynamics with the restrictions that indi- 
viduals with low scores die off and the ones with high ones flourish. More precisely, given a payoff 
matrix A we consider equations defined in the usual n— dimensional simplex of the form 

%i — X{Gi(x) 

such that 

Gi(x) > 0, if and only if (Ax)i - x l Ax > 
Gi(x) < 0, if and only if (Ax)i - x l Ax < 0. 

In this case, it follows that 

d(x) = [(Ax), - x^xjH^x) (55) 
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where Hi : ^ — > R. Moreover, form previous assumption it holds that Hi is always positive in the 
simplex ^ . We require a slightly strong condition: C + = m&x{Hi(x),x € = 1 . . . m} < +00, 
and C~ = mm{Hi (i),ie^,i = l... m} > 0. 
Then 

< C~ < fli < <7 + . (56) 

The goal is to show that a version of theorem Q] can be obtained in the present case. More precisely, 
provided the hypothesis of theorem Q] and assuming equations as above, it is shown that 



A 1 nA 



c- 



is contained in the local basin of attraction of e\. The proof, goes through the same strategy: we 
shows that for any k ^ min{-^, ^jt}, re- writing the equations in affine coordinates follows that 

Y x i°i = ^xtFiH < 

where Fi is (Ax)i — x t Ax in affine coordinates. From inequalities (|56p it follows that 

XiFi(x)Hi(x) < C + Xi Fi{x) } if Fi{x) > 0; XiF^H^x) < CxiF^x), if F^x) < 0. 

Recalling that Fj(x) = (fj — fi)(x) + R(x) with R(x) = Yliifi ~ fl)( x ) x l (the variable x is already 
assumed in affine coordinates) follows that 

Y,XiFi(x)Hi(x) < J2 C+XiFi(x)+ J2 C ~ x i F i( x ) 

i {i:Fi(x)>0} {i:Fi(x)<0} 

Y x i C + (f i -f 1 )(x)+ ^{h- h){x) 

{i:Fi(x)>0} {i:Fi(x)<0} 

+ R(x)[ y c + Xl + y c ~ x A- 



{i:F»(x)>0} {i:Fi(x)<0} 
il iuuuwb tuat Ys{i:Fi(x)>0} C + Xi + Y.{i:Fi(x)<0} ^ *U ^ ~T 



If x G A fc with k < it follows that Ylu:F 4 (x)>0\ ^ +x i + Ylu:R(x)<0\ C x i\ ^ an< ^ recaHing 
the definition of R$ follows that 

Y x iC + (fi-h)(x)+ Y XiC-(fi-fi) + R(x)[YC + Xi + Y C ~ x A^ 

{i:Fi(x)>0} {i:Fi(x)<0} 

Y x iC + (fi-h)(x)+ y x iC-(fi-h) 

{i:Fi(x)>0} {i:Fi(x)<0} 

where C + = C + — -L-, C~ = ^L-. Therefore, rewriting the equation as it was done in the proof of 
theorem [1] to finish we have to prove that 

N(cx) + x l M{cx) < (57) 

where cx = (c\Xi, C2X2, ■ ■ ■ , c n x n ) and q is either C + or C~ and N, M are the vector and matrix 
induce by A and so. To prove (|57p . we need a more general version of lemma [TUl The proofs are 
similar. 
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Lemma 19. Let c = (c±, . . . ,c m ) 6 W 11 such that each coordinate is positive. Let Q c : W 71 — > M 
given by 

Q(x) = N{cx) + x l M(cx) 

with x S R m , N £ R m , M £ R mxm and cx := (axi, ... , c m x m ). Let us assume that N t < for any 
i and for any j > i, \Ni\ ^ \Nj\. Let 

M = max{ — J , 0}. 

i,j>i — iVj 

T/ien, the set Aj_ = {x 6 ]R m : ^ 0, ^2=1 x « < aF^' * s con ^ a * nec ^ * n : Qc(^) < 0}. /n 



M 



particular, if Mq = then is treated as oo and £/iis means that {x € M m : ^ 0} C {x : 
Q c (x)^0}. 
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